2024-09-14T06:48:00.216 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T06:48:00.233 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T06:48:00.308 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408 branch: main description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900408' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: client: rbd cache: false global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5839 sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 sleep_before_teardown: 0 subset: 99/128 suite: rbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNiz1AK4+dA8lCcAIIAUsp7qbRBuuF0eIJrvNttXkv+D3S4YTAI0Fn2nUr+RGNdfM/aGH88B9MKTRUJ8AtkJkjM= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHks/XT6TB4R3tsjzw2PHsm9ZsH7tb0EA/5KHGn2x0bFxVEMLthTCe0TCq/zSVz5qbOPtEh6Ua7WVbmCtWxoaRE= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_20:16:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T06:48:00.308 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa; will attempt to use it 2024-09-14T06:48:00.309 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa/tasks 2024-09-14T06:48:00.309 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T06:48:00.311 INFO:teuthology.task.internal:Checking packages... 2024-09-14T06:48:00.333 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7f24e589328ae9e639c0009d93340e5b235d0a47' 2024-09-14T06:48:00.334 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T06:48:00.334 INFO:teuthology.packaging:ref: None 2024-09-14T06:48:00.334 INFO:teuthology.packaging:tag: None 2024-09-14T06:48:00.334 INFO:teuthology.packaging:branch: main 2024-09-14T06:48:00.334 INFO:teuthology.packaging:sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:48:00.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-09-14T06:48:00.586 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4926.g1c941a66 2024-09-14T06:48:00.588 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T06:48:00.594 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T06:48:00.594 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T06:48:00.638 INFO:teuthology.task.internal:Saving configuration 2024-09-14T06:48:00.651 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T06:48:00.664 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T06:48:00.685 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 06:42:40.747006', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNiz1AK4+dA8lCcAIIAUsp7qbRBuuF0eIJrvNttXkv+D3S4YTAI0Fn2nUr+RGNdfM/aGH88B9MKTRUJ8AtkJkjM='} 2024-09-14T06:48:00.705 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 06:42:40.745528', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHks/XT6TB4R3tsjzw2PHsm9ZsH7tb0EA/5KHGn2x0bFxVEMLthTCe0TCq/zSVz5qbOPtEh6Ua7WVbmCtWxoaRE='} 2024-09-14T06:48:00.706 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T06:48:00.712 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-14T06:48:00.712 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-14T06:48:00.712 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T06:48:00.783 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2ad2827400>, signals=[15]) 2024-09-14T06:48:00.783 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T06:48:00.790 INFO:teuthology.task.internal:Opening connections... 2024-09-14T06:48:00.790 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T06:48:00.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:48:00.885 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T06:48:00.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:48:00.961 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T06:48:00.968 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-09-14T06:48:00.986 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-09-14T06:48:00.986 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-09-14T06:48:01.044 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-09-14T06:48:01.044 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-09-14T06:48:01.044 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-09-14T06:48:01.044 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-09-14T06:48:01.044 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-09-14T06:48:01.044 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-09-14T06:48:01.044 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T06:48:01.045 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-09-14T06:48:01.045 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-09-14T06:48:01.045 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T06:48:01.045 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-09-14T06:48:01.045 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T06:48:01.045 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T06:48:01.045 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T06:48:01.046 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-09-14T06:48:01.070 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-09-14T06:48:01.087 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-09-14T06:48:01.087 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-09-14T06:48:01.144 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-09-14T06:48:01.144 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-09-14T06:48:01.144 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-09-14T06:48:01.144 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T06:48:01.145 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T06:48:01.146 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-09-14T06:48:01.178 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T06:48:01.187 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T06:48:01.213 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T06:48:01.213 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-09-14T06:48:01.217 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-09-14T06:48:01.233 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T06:48:01.238 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T06:48:01.239 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-09-14T06:48:01.274 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-09-14T06:48:01.384 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T06:48:01.433 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T06:48:01.447 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T06:48:01.447 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T06:48:01.448 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T06:48:01.448 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T06:48:01.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:48:01.465 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-14T06:48:01.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:48:01.466 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-14T06:48:01.523 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-503.el9.x86_64 2024-09-14T06:48:01.523 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-503.el9.x86_64 2024-09-14T06:48:01.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-09-14T06:48:01.525 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-503.el9.x86_64 2024-09-14T06:48:01.525 INFO:teuthology.task.kernel:Running kernel on smithi189: 5.14.0-503.el9.x86_64 2024-09-14T06:48:01.525 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum install -y kernel 2024-09-14T06:48:03.796 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-14T06:48:05.602 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.2 MB 00:02 2024-09-14T06:48:09.097 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 20 MB 00:04 2024-09-14T06:48:10.653 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 20 MB 00:03 2024-09-14T06:48:16.387 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-14T06:48:24.411 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 2.7 kB/s | 19 kB 00:06 2024-09-14T06:48:25.282 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-14T06:48:30.270 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 387 kB/s | 6.5 MB 00:17 2024-09-14T06:48:30.721 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-14T06:48:31.788 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-14T06:48:32.567 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-14T06:48:32.736 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T06:48:32.736 INFO:teuthology.orchestra.run.smithi189.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T06:48:32.840 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:48:32.845 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:48:32.845 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-14T06:48:32.845 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:48:32.845 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:48:32.845 INFO:teuthology.orchestra.run.smithi189.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T06:48:32.845 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-14T06:48:32.846 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T06:48:32.846 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T06:48:32.846 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T06:48:32.846 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:48:32.846 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:48:32.846 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:48:32.846 INFO:teuthology.orchestra.run.smithi189.stdout:Install 4 Packages 2024-09-14T06:48:32.846 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:48:32.847 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 79 M 2024-09-14T06:48:32.847 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 126 M 2024-09-14T06:48:32.847 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:48:33.039 INFO:teuthology.orchestra.run.smithi189.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 267 kB/s | 23 kB 00:00 2024-09-14T06:48:33.389 INFO:teuthology.orchestra.run.smithi189.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-14T06:48:33.815 INFO:teuthology.orchestra.run.smithi189.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 40 MB/s | 35 MB 00:00 2024-09-14T06:48:34.115 INFO:teuthology.orchestra.run.smithi189.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 27 MB/s | 29 MB 00:01 2024-09-14T06:48:34.115 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:48:34.115 INFO:teuthology.orchestra.run.smithi189.stdout:Total 62 MB/s | 79 MB 00:01 2024-09-14T06:48:34.796 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:48:35.139 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:48:35.139 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:48:36.074 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:48:36.075 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:48:38.256 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-14T06:48:38.521 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:48:39.157 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T06:48:39.241 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:48:40.363 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T06:48:40.363 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T06:48:40.452 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:48:40.458 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:48:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-14T06:48:40.458 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:48:40.458 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:48:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T06:48:40.458 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-14T06:48:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T06:48:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T06:48:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T06:48:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:48:40.459 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:48:40.459 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:48:40.459 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-14T06:48:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:48:40.460 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 79 M 2024-09-14T06:48:40.460 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 126 M 2024-09-14T06:48:40.460 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:48:40.659 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 234 kB/s | 23 kB 00:00 2024-09-14T06:48:42.050 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:48:42.186 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:48:43.385 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.5 MB/s | 16 MB 00:02 2024-09-14T06:48:45.068 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 7.7 MB/s | 35 MB 00:04 2024-09-14T06:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 5.9 MB/s | 29 MB 00:04 2024-09-14T06:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout:Total 16 MB/s | 79 MB 00:05 2024-09-14T06:48:46.177 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:48:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:48:46.570 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:48:47.039 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:48:47.137 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:48:47.545 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:48:47.546 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:48:50.829 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T06:48:50.910 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:48:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:48:53.715 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:48:53.853 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:48:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:04.173 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:30.227 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:30.491 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:30.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T06:49:30.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:49:30.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:49:31.370 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:31.370 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:49:31.370 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:49:31.370 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T06:49:31.370 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T06:49:31.370 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T06:49:31.370 INFO:teuthology.orchestra.run.smithi189.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T06:49:31.370 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:49:31.370 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:49:31.730 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T06:49:31.813 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T06:49:31.813 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T06:49:31.813 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T06:49:31.814 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T06:49:31.814 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T06:49:31.814 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T06:49:31.814 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi189.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T06:49:31.814 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T06:49:31.874 INFO:teuthology.orchestra.run.smithi189.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T06:49:31.874 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T06:49:31.874 DEBUG:teuthology.orchestra.run.smithi189:> sudo rpm -qi grub2-tools 2024-09-14T06:49:31.917 INFO:teuthology.orchestra.run.smithi189.stdout:Name : grub2-tools 2024-09-14T06:49:31.917 INFO:teuthology.orchestra.run.smithi189.stdout:Epoch : 1 2024-09-14T06:49:31.917 INFO:teuthology.orchestra.run.smithi189.stdout:Version : 2.06 2024-09-14T06:49:31.917 INFO:teuthology.orchestra.run.smithi189.stdout:Release : 82.el9 2024-09-14T06:49:31.917 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: x86_64 2024-09-14T06:49:31.917 INFO:teuthology.orchestra.run.smithi189.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T06:49:31.917 INFO:teuthology.orchestra.run.smithi189.stdout:Group : Unspecified 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:Size : 8274670 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:License : GPLv3+ 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:Packager : builder@centos.org 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:Vendor : CentOS 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:Summary : Support tools for GRUB. 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:Description : 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T06:49:31.918 INFO:teuthology.orchestra.run.smithi189.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T06:49:31.919 INFO:teuthology.orchestra.run.smithi189.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T06:49:31.919 INFO:teuthology.orchestra.run.smithi189.stdout:hardware devices. 2024-09-14T06:49:31.919 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:49:31.919 INFO:teuthology.orchestra.run.smithi189.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T06:49:31.920 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T06:49:31.920 INFO:teuthology.task.kernel:Updating grub on smithi189 to boot 5.14.0-505.el9.x86_64 2024-09-14T06:49:31.920 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T06:49:32.553 INFO:teuthology.orchestra.run.smithi189.stderr:Generating grub configuration file ... 2024-09-14T06:49:33.869 INFO:teuthology.orchestra.run.smithi189.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T06:49:33.902 INFO:teuthology.orchestra.run.smithi189.stderr:done 2024-09-14T06:49:33.905 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-09-14T06:49:33.920 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.TEVM9wWPc7 2024-09-14T06:49:33.921 DEBUG:teuthology.orchestra.run.smithi189:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.TEVM9wWPc7 2024-09-14T06:49:33.987 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.TEVM9wWPc7 2024-09-14T06:49:34.122 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.TEVM9wWPc7 is 6KB 2024-09-14T06:49:34.171 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.TEVM9wWPc7 2024-09-14T06:49:34.186 DEBUG:teuthology.orchestra.run.smithi189:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T06:49:34.249 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T06:49:34.249 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T06:49:34.249 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T06:49:34.250 INFO:teuthology.orchestra.run.smithi189.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T06:49:34.251 DEBUG:teuthology.orchestra.run.smithi189:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T06:49:34.421 DEBUG:teuthology.orchestra.run.smithi189:> sudo shutdown -r now 2024-09-14T06:49:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T06:49:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:49:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:49:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:49:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:49:44.737 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:49:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T06:49:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T06:49:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T06:49:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T06:49:44.737 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:49:44.738 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:49:45.073 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T06:49:45.164 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T06:49:45.164 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T06:49:45.164 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T06:49:45.164 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T06:49:45.164 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T06:49:45.164 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T06:49:45.164 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T06:49:45.165 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T06:49:45.223 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T06:49:45.223 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T06:49:45.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-09-14T06:49:45.260 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-09-14T06:49:45.260 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-09-14T06:49:45.260 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-09-14T06:49:45.260 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 82.el9 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8274670 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T06:49:45.261 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-09-14T06:49:45.262 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-09-14T06:49:45.262 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:49:45.262 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T06:49:45.262 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T06:49:45.262 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T06:49:45.262 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-09-14T06:49:45.262 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:49:45.262 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T06:49:45.263 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T06:49:45.263 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-505.el9.x86_64 2024-09-14T06:49:45.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T06:49:45.948 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-09-14T06:49:47.233 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T06:49:47.259 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-09-14T06:49:47.262 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-14T06:49:47.277 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.R1qNbljkr0 2024-09-14T06:49:47.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.R1qNbljkr0 2024-09-14T06:49:47.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.R1qNbljkr0 2024-09-14T06:49:47.481 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.R1qNbljkr0 is 6KB 2024-09-14T06:49:47.530 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.R1qNbljkr0 2024-09-14T06:49:47.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T06:49:47.607 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T06:49:47.607 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T06:49:47.607 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T06:49:47.607 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T06:49:47.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T06:49:47.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-09-14T06:50:04.441 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T06:50:04.441 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-14T06:50:04.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:50:17.777 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T06:50:17.777 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-14T06:50:17.778 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:50:22.909 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-14T06:50:31.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-14T06:50:31.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:50:34.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-14T06:50:36.221 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-14T06:50:45.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-14T06:50:45.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:50:46.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-14T06:50:46.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:51:45.290 DEBUG:teuthology.orchestra.remote:timed out 2024-09-14T06:51:46.980 DEBUG:teuthology.orchestra.remote:timed out 2024-09-14T06:51:57.301 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-14T06:51:57.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:51:57.583 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-14T06:51:57.947 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-14T06:51:57.948 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-14T06:51:57.948 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-14T06:51:57.965 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-505.el9.x86_64 2024-09-14T06:51:57.965 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T06:51:57.965 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T06:51:57.965 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T06:51:58.966 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-14T06:51:58.967 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T06:51:59.039 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2024-09-14T06:51:59.059 DEBUG:teuthology.parallel:result is None 2024-09-14T06:52:01.981 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-14T06:52:01.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:52:02.337 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-14T06:52:02.691 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-14T06:52:02.691 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-14T06:52:02.691 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-14T06:52:02.707 INFO:teuthology.orchestra.run.smithi189.stdout:5.14.0-505.el9.x86_64 2024-09-14T06:52:02.707 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T06:52:02.707 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T06:52:02.707 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T06:52:03.708 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-14T06:52:03.709 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T06:52:03.800 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-09-14T06:52:03.817 DEBUG:teuthology.parallel:result is None 2024-09-14T06:52:03.818 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-14T06:52:03.824 INFO:teuthology.task.internal:Creating test directory... 2024-09-14T06:52:03.825 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T06:52:03.827 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T06:52:03.846 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-14T06:52:03.854 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-14T06:52:03.888 INFO:teuthology.task.internal:Creating archive directory... 2024-09-14T06:52:03.888 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T06:52:03.891 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T06:52:03.924 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-14T06:52:03.930 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-14T06:52:03.930 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T06:52:03.957 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T06:52:03.990 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:52:03.998 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:52:04.002 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:52:04.010 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:52:04.011 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-14T06:52:04.019 INFO:teuthology.task.internal:Configuring sudo... 2024-09-14T06:52:04.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T06:52:04.046 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T06:52:04.079 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-14T06:52:04.088 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-14T06:52:04.088 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T06:52:04.116 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T06:52:04.135 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:52:04.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:52:04.263 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:52:04.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:52:04.389 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:52:04.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T06:52:04.456 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:52:04.500 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:52:04.551 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:52:04.634 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:52:04.676 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:52:04.676 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T06:52:04.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-09-14T06:52:04.744 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-09-14T06:52:04.790 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T06:52:04.825 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T06:52:05.317 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-14T06:52:05.324 INFO:teuthology.task.internal:Starting timer... 2024-09-14T06:52:05.324 INFO:teuthology.run_tasks:Running task pcp... 2024-09-14T06:52:05.367 INFO:teuthology.run_tasks:Running task selinux... 2024-09-14T06:52:05.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-09-14T06:52:05.446 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-09-14T06:52:05.448 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-09-14T06:52:05.487 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-09-14T06:52:05.489 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-14T06:52:05.489 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-09-14T06:52:05.513 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-09-14T06:52:05.513 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-09-14T06:52:05.594 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-09-14T06:52:05.595 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-09-14T06:52:05.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T06:52:05.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:52:05.625 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T06:52:05.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:52:05.651 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-14T06:52:05.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-09-14T06:52:05.701 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-09-14T06:52:05.738 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-14T06:52:05.747 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-14T06:52:05.759 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-14T06:52:05.759 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-14T06:55:51.038 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-09-14T06:55:51.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-14T06:55:51.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:55:51.130 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-14T06:55:51.210 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-14T06:55:51.211 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-14T06:55:51.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:55:51.288 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-14T06:55:51.365 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-14T06:55:51.365 INFO:teuthology.run_tasks:Running task clock... 2024-09-14T06:55:51.377 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-14T06:55:51.377 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T06:55:51.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:55:51.380 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T06:55:51.380 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:55:51.413 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T06:55:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T06:55:51.435 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T06:55:51.450 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T06:55:51.458 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-09-14T06:55:51.470 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-09-14T06:55:51.475 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-09-14T06:55:51.486 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-09-14T06:55:51.487 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T06:55:51.501 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T06:55:51.502 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T06:55:51.515 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T06:55:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-14T06:55:51.565 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T06:55:51.565 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-14T06:55:51.565 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:55:51.565 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:55:51.565 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:55:51.565 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:55:51.575 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-14T06:55:51.579 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T06:55:51.579 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-14T06:55:51.579 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:55:51.579 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:55:51.579 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:55:51.580 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:55:51.580 INFO:teuthology.run_tasks:Running task install... 2024-09-14T06:55:51.588 DEBUG:teuthology.task.install:project ceph 2024-09-14T06:55:51.588 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}} 2024-09-14T06:55:51.588 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'} 2024-09-14T06:55:51.588 INFO:teuthology.task.install:Using flavor: default 2024-09-14T06:55:51.594 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-14T06:55:51.594 INFO:teuthology.task.install:extra packages: [] 2024-09-14T06:55:51.594 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'tag': None, 'wait_for_package': False} 2024-09-14T06:55:51.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:55:51.597 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'tag': None, 'wait_for_package': False} 2024-09-14T06:55:51.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:55:51.747 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/ 2024-09-14T06:55:51.747 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4775.g7f24e589 2024-09-14T06:55:51.752 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/ 2024-09-14T06:55:51.752 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4775.g7f24e589 2024-09-14T06:55:51.860 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T06:55:51.860 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:55:51.860 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T06:55:51.867 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T06:55:51.867 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:55:51.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T06:55:51.887 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T06:55:51.887 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7f24e589328ae9e639c0009d93340e5b235d0a47/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T06:55:51.895 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T06:55:51.895 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7f24e589328ae9e639c0009d93340e5b235d0a47/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T06:55:51.956 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T06:55:51.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T06:55:52.028 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T06:55:52.040 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T06:55:52.092 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-09-14T06:55:52.093 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-09-14T06:55:52.105 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-09-14T06:55:52.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-09-14T06:55:52.320 INFO:teuthology.orchestra.run.smithi189.stdout:45 files removed 2024-09-14T06:55:52.345 INFO:teuthology.orchestra.run.smithi007.stdout:45 files removed 2024-09-14T06:55:52.347 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw 2024-09-14T06:55:52.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw 2024-09-14T06:55:52.912 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 300 kB/s | 84 kB 00:00 2024-09-14T06:55:52.936 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 318 kB/s | 84 kB 00:00 2024-09-14T06:55:53.190 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 103 kB/s | 24 kB 00:00 2024-09-14T06:55:53.237 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-14T06:55:53.465 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-14T06:55:53.512 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-14T06:55:54.107 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-14T06:55:55.595 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.2 MB 00:02 2024-09-14T06:55:56.275 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-14T06:55:57.347 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-14T06:56:00.826 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.5 MB 00:00 2024-09-14T06:56:01.706 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-14T06:56:02.385 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-14T06:56:03.130 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-14T06:56:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-14T06:56:04.011 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-14T06:56:08.644 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-14T06:56:09.683 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-14T06:56:11.030 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:56:11.048 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:56:11.048 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:56:11.048 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:56:11.048 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:56:11.048 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 11 M 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.5 M 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 22 M 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 25 k 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T06:56:11.049 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 755 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 508 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 45 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 143 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 161 k 2024-09-14T06:56:11.050 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 321 k 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 306 k 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 99 k 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout:Install 31 Packages 2024-09-14T06:56:11.051 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 2 Packages 2024-09-14T06:56:11.052 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:11.052 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 62 M 2024-09-14T06:56:11.052 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:56:11.954 INFO:teuthology.orchestra.run.smithi189.stdout:(1/33): ceph-base-19.3.0-4775.g7f24e589.el9.x86 13 MB/s | 5.5 MB 00:00 2024-09-14T06:56:11.996 INFO:teuthology.orchestra.run.smithi189.stdout:(2/33): ceph-selinux-19.3.0-4775.g7f24e589.el9. 607 kB/s | 25 kB 00:00 2024-09-14T06:56:12.080 INFO:teuthology.orchestra.run.smithi189.stdout:(3/33): libcephfs2-19.3.0-4775.g7f24e589.el9.x8 8.9 MB/s | 755 kB 00:00 2024-09-14T06:56:12.154 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:56:12.163 INFO:teuthology.orchestra.run.smithi189.stdout:(4/33): libradosstriper1-19.3.0-4775.g7f24e589. 6.0 MB/s | 508 kB 00:00 2024-09-14T06:56:12.166 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:56:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:56:12.166 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:56:12.166 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:56:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 11 M 2024-09-14T06:56:12.166 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-09-14T06:56:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.5 M 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 22 M 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 25 k 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 755 k 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T06:56:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 508 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 45 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 143 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 161 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 321 k 2024-09-14T06:56:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 306 k 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 99 k 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout:Install 31 Packages 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 2 Packages 2024-09-14T06:56:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:12.170 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 62 M 2024-09-14T06:56:12.170 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:56:12.513 INFO:teuthology.orchestra.run.smithi189.stdout:(5/33): ceph-radosgw-19.3.0-4775.g7f24e589.el9. 11 MB/s | 11 MB 00:00 2024-09-14T06:56:12.597 INFO:teuthology.orchestra.run.smithi189.stdout:(6/33): librgw2-19.3.0-4775.g7f24e589.el9.x86_6 13 MB/s | 5.5 MB 00:00 2024-09-14T06:56:12.614 INFO:teuthology.orchestra.run.smithi189.stdout:(7/33): python3-ceph-argparse-19.3.0-4775.g7f24 451 kB/s | 45 kB 00:00 2024-09-14T06:56:12.648 INFO:teuthology.orchestra.run.smithi189.stdout:(8/33): python3-ceph-common-19.3.0-4775.g7f24e5 2.8 MB/s | 143 kB 00:00 2024-09-14T06:56:12.672 INFO:teuthology.orchestra.run.smithi189.stdout:(9/33): python3-cephfs-19.3.0-4775.g7f24e589.el 2.7 MB/s | 161 kB 00:00 2024-09-14T06:56:12.698 INFO:teuthology.orchestra.run.smithi189.stdout:(10/33): python3-rados-19.3.0-4775.g7f24e589.el 6.2 MB/s | 321 kB 00:00 2024-09-14T06:56:12.723 INFO:teuthology.orchestra.run.smithi189.stdout:(11/33): python3-rbd-19.3.0-4775.g7f24e589.el9. 5.9 MB/s | 306 kB 00:00 2024-09-14T06:56:12.748 INFO:teuthology.orchestra.run.smithi189.stdout:(12/33): python3-rgw-19.3.0-4775.g7f24e589.el9. 1.9 MB/s | 99 kB 00:00 2024-09-14T06:56:12.951 INFO:teuthology.orchestra.run.smithi007.stdout:(1/33): ceph-base-19.3.0-4775.g7f24e589.el9.x86 14 MB/s | 5.5 MB 00:00 2024-09-14T06:56:12.966 INFO:teuthology.orchestra.run.smithi189.stdout:(13/33): ceph-common-19.3.0-4775.g7f24e589.el9. 15 MB/s | 22 MB 00:01 2024-09-14T06:56:13.010 INFO:teuthology.orchestra.run.smithi007.stdout:(2/33): ceph-selinux-19.3.0-4775.g7f24e589.el9. 433 kB/s | 25 kB 00:00 2024-09-14T06:56:13.015 INFO:teuthology.orchestra.run.smithi189.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 366 kB/s | 107 kB 00:00 2024-09-14T06:56:13.066 INFO:teuthology.orchestra.run.smithi189.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 538 kB/s | 171 kB 00:00 2024-09-14T06:56:13.102 INFO:teuthology.orchestra.run.smithi007.stdout:(3/33): libcephfs2-19.3.0-4775.g7f24e589.el9.x8 8.0 MB/s | 755 kB 00:00 2024-09-14T06:56:13.108 INFO:teuthology.orchestra.run.smithi189.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 493 kB/s | 45 kB 00:00 2024-09-14T06:56:13.133 INFO:teuthology.orchestra.run.smithi189.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 959 kB/s | 160 kB 00:00 2024-09-14T06:56:13.175 INFO:teuthology.orchestra.run.smithi189.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 662 kB 00:00 2024-09-14T06:56:13.194 INFO:teuthology.orchestra.run.smithi007.stdout:(4/33): libradosstriper1-19.3.0-4775.g7f24e589. 5.4 MB/s | 508 kB 00:00 2024-09-14T06:56:13.217 INFO:teuthology.orchestra.run.smithi189.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 505 kB/s | 42 kB 00:00 2024-09-14T06:56:13.259 INFO:teuthology.orchestra.run.smithi189.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-14T06:56:13.284 INFO:teuthology.orchestra.run.smithi189.stdout:(21/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-14T06:56:13.318 INFO:teuthology.orchestra.run.smithi189.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 358 kB/s | 36 kB 00:00 2024-09-14T06:56:13.343 INFO:teuthology.orchestra.run.smithi189.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-14T06:56:13.384 INFO:teuthology.orchestra.run.smithi189.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-14T06:56:13.410 INFO:teuthology.orchestra.run.smithi189.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-14T06:56:13.435 INFO:teuthology.orchestra.run.smithi189.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 307 kB/s | 26 kB 00:00 2024-09-14T06:56:13.460 INFO:teuthology.orchestra.run.smithi189.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-14T06:56:13.494 INFO:teuthology.orchestra.run.smithi189.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-14T06:56:13.544 INFO:teuthology.orchestra.run.smithi007.stdout:(5/33): ceph-radosgw-19.3.0-4775.g7f24e589.el9. 11 MB/s | 11 MB 00:00 2024-09-14T06:56:13.561 INFO:teuthology.orchestra.run.smithi189.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-09-14T06:56:13.611 INFO:teuthology.orchestra.run.smithi189.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-14T06:56:13.627 INFO:teuthology.orchestra.run.smithi007.stdout:(6/33): librgw2-19.3.0-4775.g7f24e589.el9.x86_6 13 MB/s | 5.5 MB 00:00 2024-09-14T06:56:13.653 INFO:teuthology.orchestra.run.smithi007.stdout:(7/33): python3-ceph-argparse-19.3.0-4775.g7f24 415 kB/s | 45 kB 00:00 2024-09-14T06:56:13.678 INFO:teuthology.orchestra.run.smithi007.stdout:(8/33): python3-ceph-common-19.3.0-4775.g7f24e5 2.8 MB/s | 143 kB 00:00 2024-09-14T06:56:13.695 INFO:teuthology.orchestra.run.smithi007.stdout:(9/33): python3-cephfs-19.3.0-4775.g7f24e589.el 3.8 MB/s | 161 kB 00:00 2024-09-14T06:56:13.712 INFO:teuthology.orchestra.run.smithi189.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-14T06:56:13.737 INFO:teuthology.orchestra.run.smithi007.stdout:(10/33): python3-rados-19.3.0-4775.g7f24e589.el 5.4 MB/s | 321 kB 00:00 2024-09-14T06:56:13.762 INFO:teuthology.orchestra.run.smithi007.stdout:(11/33): python3-rbd-19.3.0-4775.g7f24e589.el9. 4.5 MB/s | 306 kB 00:00 2024-09-14T06:56:13.829 INFO:teuthology.orchestra.run.smithi007.stdout:(12/33): python3-rgw-19.3.0-4775.g7f24e589.el9. 1.1 MB/s | 99 kB 00:00 2024-09-14T06:56:13.845 INFO:teuthology.orchestra.run.smithi189.stdout:(32/33): librados2-19.3.0-4775.g7f24e589.el9.x8 12 MB/s | 3.5 MB 00:00 2024-09-14T06:56:13.912 INFO:teuthology.orchestra.run.smithi189.stdout:(33/33): librbd1-19.3.0-4775.g7f24e589.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-14T06:56:13.917 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:56:13.917 INFO:teuthology.orchestra.run.smithi189.stdout:Total 22 MB/s | 62 MB 00:02 2024-09-14T06:56:14.034 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:56:14.038 INFO:teuthology.orchestra.run.smithi007.stdout:(13/33): ceph-common-19.3.0-4775.g7f24e589.el9. 15 MB/s | 22 MB 00:01 2024-09-14T06:56:14.063 INFO:teuthology.orchestra.run.smithi007.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 356 kB/s | 107 kB 00:00 2024-09-14T06:56:14.119 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:56:14.119 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:56:14.122 INFO:teuthology.orchestra.run.smithi007.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-09-14T06:56:14.147 INFO:teuthology.orchestra.run.smithi007.stdout:(16/33): libnbd-1.20.2-2.el9.x86_64.rpm 537 kB/s | 171 kB 00:00 2024-09-14T06:56:14.172 INFO:teuthology.orchestra.run.smithi007.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-14T06:56:14.214 INFO:teuthology.orchestra.run.smithi007.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-14T06:56:14.239 INFO:teuthology.orchestra.run.smithi007.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 631 kB/s | 42 kB 00:00 2024-09-14T06:56:14.265 INFO:teuthology.orchestra.run.smithi007.stdout:(20/33): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-14T06:56:14.290 INFO:teuthology.orchestra.run.smithi007.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 714 kB/s | 36 kB 00:00 2024-09-14T06:56:14.316 INFO:teuthology.orchestra.run.smithi007.stdout:(22/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.3 MB/s | 662 kB 00:00 2024-09-14T06:56:14.341 INFO:teuthology.orchestra.run.smithi007.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-14T06:56:14.399 INFO:teuthology.orchestra.run.smithi007.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-14T06:56:14.416 INFO:teuthology.orchestra.run.smithi007.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2024-09-14T06:56:14.433 INFO:teuthology.orchestra.run.smithi007.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 341 kB/s | 26 kB 00:00 2024-09-14T06:56:14.459 INFO:teuthology.orchestra.run.smithi007.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-14T06:56:14.492 INFO:teuthology.orchestra.run.smithi007.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-14T06:56:14.543 INFO:teuthology.orchestra.run.smithi007.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-14T06:56:14.593 INFO:teuthology.orchestra.run.smithi007.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-14T06:56:14.668 INFO:teuthology.orchestra.run.smithi007.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-14T06:56:14.803 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:56:14.803 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:56:14.827 INFO:teuthology.orchestra.run.smithi007.stdout:(32/33): librados2-19.3.0-4775.g7f24e589.el9.x8 12 MB/s | 3.5 MB 00:00 2024-09-14T06:56:14.877 INFO:teuthology.orchestra.run.smithi007.stdout:(33/33): librbd1-19.3.0-4775.g7f24e589.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-14T06:56:14.882 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:56:14.882 INFO:teuthology.orchestra.run.smithi007.stdout:Total 23 MB/s | 62 MB 00:02 2024-09-14T06:56:14.999 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:56:15.039 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:56:15.039 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:56:15.728 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:56:15.728 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:56:16.361 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:56:16.432 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-14T06:56:16.466 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-14T06:56:16.492 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-14T06:56:16.515 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T06:56:16.542 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T06:56:16.700 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T06:56:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T06:56:17.099 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-14T06:56:17.202 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:56:17.287 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:56:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:56:17.374 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 8/35 2024-09-14T06:56:17.395 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T06:56:17.400 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-14T06:56:17.433 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-14T06:56:17.441 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T06:56:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-14T06:56:17.479 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-14T06:56:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T06:56:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T06:56:17.514 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 11/35 2024-09-14T06:56:17.583 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T06:56:17.605 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T06:56:17.670 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T06:56:17.700 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T06:56:17.704 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-14T06:56:17.753 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T06:56:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-14T06:56:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-14T06:56:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-14T06:56:17.920 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-14T06:56:17.949 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-14T06:56:17.982 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T06:56:18.019 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-14T06:56:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-14T06:56:18.242 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:56:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:56:18.341 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-14T06:56:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T06:56:18.425 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 8/35 2024-09-14T06:56:18.452 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T06:56:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T06:56:18.492 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 24/35 2024-09-14T06:56:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T06:56:18.528 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-14T06:56:18.544 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-14T06:56:18.588 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 11/35 2024-09-14T06:56:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T06:56:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T06:56:18.761 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T06:56:18.803 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-14T06:56:18.922 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-14T06:56:19.023 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-14T06:56:19.056 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T06:56:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-14T06:56:19.127 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T06:56:19.153 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-14T06:56:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-14T06:56:19.205 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 28/35 2024-09-14T06:56:19.216 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-14T06:56:19.233 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 29/35 2024-09-14T06:56:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T06:56:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-14T06:56:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-14T06:56:19.662 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T06:56:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T06:56:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 24/35 2024-09-14T06:56:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-14T06:56:20.362 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-14T06:56:20.390 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T06:56:20.462 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T06:56:20.542 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 28/35 2024-09-14T06:56:20.595 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 29/35 2024-09-14T06:56:21.603 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:56:21.638 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:56:22.230 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:56:22.266 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T06:56:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T06:56:22.362 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T06:56:22.362 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T06:56:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:56:23.014 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:56:23.064 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:56:23.671 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:56:23.709 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T06:56:23.800 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T06:56:23.800 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T06:56:23.801 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T06:56:23.801 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:56:32.413 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:56:32.413 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-09-14T06:56:32.414 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-09-14T06:56:32.414 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-09-14T06:56:32.414 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-09-14T06:56:32.414 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-09-14T06:56:32.414 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-09-14T06:56:32.414 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-09-14T06:56:32.414 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:33.408 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T06:56:33.433 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T06:56:33.433 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:56:33.433 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T06:56:33.433 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:56:33.434 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:56:33.434 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:33.435 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T06:56:33.518 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T06:56:33.518 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:56:34.620 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:56:34.620 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-09-14T06:56:34.620 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-09-14T06:56:34.621 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-09-14T06:56:34.621 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-09-14T06:56:34.621 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-09-14T06:56:34.621 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-09-14T06:56:34.621 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-09-14T06:56:34.621 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:56:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 1/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 3/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 4/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 5/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 6/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 8/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 9/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 10/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 11/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T06:56:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 13/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-14T06:56:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-14T06:56:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-14T06:56:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-14T06:56:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-14T06:56:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-14T06:56:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:56:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-14T06:56:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 34/35 2024-09-14T06:56:35.599 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:56:35.599 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.600 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T06:56:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T06:56:35.602 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T06:56:35.603 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T06:56:35.603 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T06:56:35.603 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T06:56:35.603 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:35.603 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:56:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T06:56:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T06:56:35.725 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:56:35.725 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T06:56:35.725 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:56:35.725 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:56:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T06:56:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T06:56:35.826 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:56:35.962 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-test 2024-09-14T06:56:36.536 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:28 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:56:36.669 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:56:36.669 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:56:36.669 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 50 M 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 162 k 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout:Install 4 Packages 2024-09-14T06:56:36.670 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:36.671 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 51 M 2024-09-14T06:56:36.671 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 217 M 2024-09-14T06:56:36.671 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:56:36.823 INFO:teuthology.orchestra.run.smithi189.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.3 MB/s | 304 kB 00:00 2024-09-14T06:56:36.865 INFO:teuthology.orchestra.run.smithi189.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-14T06:56:36.906 INFO:teuthology.orchestra.run.smithi189.stdout:(3/4): libcephsqlite-19.3.0-4775.g7f24e589.el9. 759 kB/s | 162 kB 00:00 2024-09-14T06:56:37.073 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 1/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 3/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 4/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 5/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 6/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 8/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 9/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 10/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 11/35 2024-09-14T06:56:37.074 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 13/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-14T06:56:37.075 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-14T06:56:37.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-14T06:56:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-14T06:56:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-14T06:56:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-14T06:56:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-14T06:56:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:56:37.077 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-14T06:56:37.078 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 34/35 2024-09-14T06:56:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:56:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:37.969 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-09-14T06:56:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.969 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T06:56:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T06:56:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T06:56:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T06:56:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T06:56:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T06:56:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:37.972 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:56:38.257 INFO:teuthology.orchestra.run.smithi189.stdout:(4/4): ceph-test-19.3.0-4775.g7f24e589.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-14T06:56:38.259 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:56:38.259 INFO:teuthology.orchestra.run.smithi189.stdout:Total 32 MB/s | 51 MB 00:01 2024-09-14T06:56:38.286 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:56:38.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-test 2024-09-14T06:56:38.307 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:56:38.307 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:56:38.706 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:56:38.707 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:56:38.887 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:29 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:56:39.033 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 50 M 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 162 k 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:56:39.034 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-14T06:56:39.035 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:39.035 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 51 M 2024-09-14T06:56:39.035 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 217 M 2024-09-14T06:56:39.035 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:56:39.247 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:56:39.326 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T06:56:39.364 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T06:56:39.392 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T06:56:39.431 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.6 MB/s | 304 kB 00:00 2024-09-14T06:56:39.465 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-14T06:56:39.524 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): libcephsqlite-19.3.0-4775.g7f24e589.el9. 779 kB/s | 162 kB 00:00 2024-09-14T06:56:40.916 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): ceph-test-19.3.0-4775.g7f24e589.el9.x86_ 31 MB/s | 50 MB 00:01 2024-09-14T06:56:40.918 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:56:40.918 INFO:teuthology.orchestra.run.smithi007.stdout:Total 27 MB/s | 51 MB 00:01 2024-09-14T06:56:40.946 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:56:40.969 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:56:40.970 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:56:41.382 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:56:41.382 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:56:41.936 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:56:42.037 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T06:56:42.082 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T06:56:42.109 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T06:56:44.154 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T06:56:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T06:56:46.621 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T06:56:46.621 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 1/4 2024-09-14T06:56:46.622 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2/4 2024-09-14T06:56:46.622 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T06:56:46.954 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T06:56:46.954 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:46.954 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:56:46.955 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:46.955 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:46.955 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T06:56:46.955 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T06:56:46.955 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:46.955 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:56:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T06:56:47.202 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T06:56:47.222 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph 2024-09-14T06:56:47.805 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:39 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:56:47.955 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 6.6 k 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 2.4 M 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 1.5 M 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 291 k 2024-09-14T06:56:47.957 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 4.8 M 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 17 M 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections 2024-09-14T06:56:47.958 INFO:teuthology.orchestra.run.smithi189.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T06:56:47.959 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout:Install 37 Packages 2024-09-14T06:56:47.960 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:47.961 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 31 M 2024-09-14T06:56:47.961 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 109 M 2024-09-14T06:56:47.961 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:56:48.523 INFO:teuthology.orchestra.run.smithi189.stdout:(1/37): ceph-19.3.0-4775.g7f24e589.el9.x86_64.r 51 kB/s | 6.6 kB 00:00 2024-09-14T06:56:48.665 INFO:teuthology.orchestra.run.smithi189.stdout:(2/37): ceph-mgr-19.3.0-4775.g7f24e589.el9.x86_ 5.6 MB/s | 1.5 MB 00:00 2024-09-14T06:56:48.748 INFO:teuthology.orchestra.run.smithi189.stdout:(3/37): ceph-mds-19.3.0-4775.g7f24e589.el9.x86_ 6.7 MB/s | 2.4 MB 00:00 2024-09-14T06:56:48.832 INFO:teuthology.orchestra.run.smithi189.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4775.g7f24 3.4 MB/s | 291 kB 00:00 2024-09-14T06:56:48.899 INFO:teuthology.orchestra.run.smithi189.stdout:(5/37): ceph-mon-19.3.0-4775.g7f24e589.el9.x86_ 13 MB/s | 4.8 MB 00:00 2024-09-14T06:56:48.999 INFO:teuthology.orchestra.run.smithi189.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-09-14T06:56:49.050 INFO:teuthology.orchestra.run.smithi189.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-14T06:56:49.092 INFO:teuthology.orchestra.run.smithi189.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 6.4 MB/s | 1.2 MB 00:00 2024-09-14T06:56:49.117 INFO:teuthology.orchestra.run.smithi189.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-14T06:56:49.142 INFO:teuthology.orchestra.run.smithi189.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-14T06:56:49.167 INFO:teuthology.orchestra.run.smithi189.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-14T06:56:49.360 INFO:teuthology.orchestra.run.smithi189.stdout:(12/37): ceph-osd-19.3.0-4775.g7f24e589.el9.x86 25 MB/s | 17 MB 00:00 2024-09-14T06:56:49.401 INFO:teuthology.orchestra.run.smithi189.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 727 kB/s | 188 kB 00:00 2024-09-14T06:56:49.468 INFO:teuthology.orchestra.run.smithi189.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 572 kB/s | 172 kB 00:00 2024-09-14T06:56:49.497 INFO:teuthology.orchestra.run.smithi189.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 260 kB/s | 35 kB 00:00 2024-09-14T06:56:49.519 INFO:teuthology.orchestra.run.smithi189.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 658 kB/s | 77 kB 00:00 2024-09-14T06:56:49.536 INFO:teuthology.orchestra.run.smithi189.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-09-14T06:56:49.578 INFO:teuthology.orchestra.run.smithi189.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 497 kB/s | 42 kB 00:00 2024-09-14T06:56:49.611 INFO:teuthology.orchestra.run.smithi189.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-14T06:56:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T06:56:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 1/4 2024-09-14T06:56:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2/4 2024-09-14T06:56:49.613 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T06:56:49.628 INFO:teuthology.orchestra.run.smithi189.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 204 kB/s | 22 kB 00:00 2024-09-14T06:56:49.645 INFO:teuthology.orchestra.run.smithi189.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 647 kB/s | 43 kB 00:00 2024-09-14T06:56:49.670 INFO:teuthology.orchestra.run.smithi189.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 598 kB/s | 11 kB 00:00 2024-09-14T06:56:49.704 INFO:teuthology.orchestra.run.smithi189.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.9 MB/s | 172 kB 00:00 2024-09-14T06:56:49.729 INFO:teuthology.orchestra.run.smithi189.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 311 kB/s | 18 kB 00:00 2024-09-14T06:56:49.766 INFO:teuthology.orchestra.run.smithi189.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 375 kB/s | 23 kB 00:00 2024-09-14T06:56:49.791 INFO:teuthology.orchestra.run.smithi189.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 316 kB/s | 19 kB 00:00 2024-09-14T06:56:49.816 INFO:teuthology.orchestra.run.smithi189.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 394 kB/s | 20 kB 00:00 2024-09-14T06:56:49.842 INFO:teuthology.orchestra.run.smithi189.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 928 kB/s | 46 kB 00:00 2024-09-14T06:56:49.867 INFO:teuthology.orchestra.run.smithi189.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-14T06:56:49.892 INFO:teuthology.orchestra.run.smithi189.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 660 kB/s | 16 kB 00:00 2024-09-14T06:56:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T06:56:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:49.914 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:56:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T06:56:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T06:56:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:49.915 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:56:49.917 INFO:teuthology.orchestra.run.smithi189.stdout:(31/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-09-14T06:56:49.951 INFO:teuthology.orchestra.run.smithi189.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-14T06:56:49.977 INFO:teuthology.orchestra.run.smithi189.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 609 kB/s | 36 kB 00:00 2024-09-14T06:56:50.002 INFO:teuthology.orchestra.run.smithi189.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-14T06:56:50.027 INFO:teuthology.orchestra.run.smithi189.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 806 kB/s | 20 kB 00:00 2024-09-14T06:56:50.052 INFO:teuthology.orchestra.run.smithi189.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-14T06:56:50.078 INFO:teuthology.orchestra.run.smithi189.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-14T06:56:50.082 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:56:50.082 INFO:teuthology.orchestra.run.smithi189.stdout:Total 14 MB/s | 31 MB 00:02 2024-09-14T06:56:50.178 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph 2024-09-14T06:56:50.233 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:56:50.273 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:56:50.274 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:56:50.765 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:41 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:56:50.774 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:56:50.774 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:56:50.923 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:56:50.925 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:56:50.925 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 6.6 k 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 2.4 M 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 1.5 M 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 291 k 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 4.8 M 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 17 M 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T06:56:50.926 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T06:56:50.927 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T06:56:50.928 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout:Install 37 Packages 2024-09-14T06:56:50.929 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:50.930 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 31 M 2024-09-14T06:56:50.930 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 109 M 2024-09-14T06:56:50.930 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:56:51.276 INFO:teuthology.orchestra.run.smithi007.stdout:(1/37): ceph-19.3.0-4775.g7f24e589.el9.x86_64.r 50 kB/s | 6.6 kB 00:00 2024-09-14T06:56:51.477 INFO:teuthology.orchestra.run.smithi007.stdout:(2/37): ceph-mgr-19.3.0-4775.g7f24e589.el9.x86_ 4.6 MB/s | 1.5 MB 00:00 2024-09-14T06:56:51.527 INFO:teuthology.orchestra.run.smithi007.stdout:(3/37): ceph-mds-19.3.0-4775.g7f24e589.el9.x86_ 6.3 MB/s | 2.4 MB 00:00 2024-09-14T06:56:51.611 INFO:teuthology.orchestra.run.smithi007.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4775.g7f24 3.4 MB/s | 291 kB 00:00 2024-09-14T06:56:51.711 INFO:teuthology.orchestra.run.smithi007.stdout:(5/37): ceph-mon-19.3.0-4775.g7f24e589.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-14T06:56:52.036 INFO:teuthology.orchestra.run.smithi007.stdout:(6/37): ceph-osd-19.3.0-4775.g7f24e589.el9.x86_ 31 MB/s | 17 MB 00:00 2024-09-14T06:56:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:56:52.172 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T06:56:52.213 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T06:56:52.274 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T06:56:52.345 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T06:56:52.390 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T06:56:52.423 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T06:56:52.516 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T06:56:52.578 INFO:teuthology.orchestra.run.smithi007.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 262 kB/s | 253 kB 00:00 2024-09-14T06:56:52.594 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T06:56:52.634 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T06:56:52.669 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T06:56:52.698 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T06:56:52.733 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T06:56:52.766 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T06:56:52.807 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T06:56:52.811 INFO:teuthology.orchestra.run.smithi007.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 139 kB/s | 106 kB 00:00 2024-09-14T06:56:52.842 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T06:56:52.876 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T06:56:52.924 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T06:56:52.965 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T06:56:53.038 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T06:56:53.187 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T06:56:53.264 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T06:56:53.312 INFO:teuthology.orchestra.run.smithi007.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 184 kB/s | 135 kB 00:00 2024-09-14T06:56:53.317 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T06:56:53.355 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T06:56:53.405 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T06:56:53.467 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T06:56:53.587 INFO:teuthology.orchestra.run.smithi007.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 163 kB/s | 125 kB 00:00 2024-09-14T06:56:53.662 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T06:56:53.709 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T06:56:53.771 INFO:teuthology.orchestra.run.smithi007.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 609 kB/s | 1.2 MB 00:02 2024-09-14T06:56:53.787 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T06:56:53.876 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T06:56:53.896 INFO:teuthology.orchestra.run.smithi007.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 610 kB/s | 188 kB 00:00 2024-09-14T06:56:53.916 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T06:56:53.938 INFO:teuthology.orchestra.run.smithi007.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 837 kB/s | 35 kB 00:00 2024-09-14T06:56:53.980 INFO:teuthology.orchestra.run.smithi007.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 826 kB/s | 172 kB 00:00 2024-09-14T06:56:54.005 INFO:teuthology.orchestra.run.smithi007.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.1 MB/s | 77 kB 00:00 2024-09-14T06:56:54.030 INFO:teuthology.orchestra.run.smithi007.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-14T06:56:54.045 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T06:56:54.047 INFO:teuthology.orchestra.run.smithi007.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 998 kB/s | 42 kB 00:00 2024-09-14T06:56:54.073 INFO:teuthology.orchestra.run.smithi007.stdout:(18/37): python3-urllib3-1.26.5-6.el9.noarch.rp 282 kB/s | 215 kB 00:00 2024-09-14T06:56:54.098 INFO:teuthology.orchestra.run.smithi007.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 332 kB/s | 22 kB 00:00 2024-09-14T06:56:54.148 INFO:teuthology.orchestra.run.smithi007.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-14T06:56:54.165 INFO:teuthology.orchestra.run.smithi007.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 517 kB/s | 43 kB 00:00 2024-09-14T06:56:54.199 INFO:teuthology.orchestra.run.smithi007.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 322 kB/s | 11 kB 00:00 2024-09-14T06:56:54.206 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 32/37 2024-09-14T06:56:54.215 INFO:teuthology.orchestra.run.smithi007.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 1.0 MB/s | 18 kB 00:00 2024-09-14T06:56:54.235 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T06:56:54.241 INFO:teuthology.orchestra.run.smithi007.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-14T06:56:54.258 INFO:teuthology.orchestra.run.smithi007.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 553 kB/s | 23 kB 00:00 2024-09-14T06:56:54.261 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T06:56:54.261 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:56:54.261 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T06:56:54.261 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T06:56:54.261 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T06:56:54.262 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:54.275 INFO:teuthology.orchestra.run.smithi007.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 578 kB/s | 19 kB 00:00 2024-09-14T06:56:54.292 INFO:teuthology.orchestra.run.smithi007.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 583 kB/s | 20 kB 00:00 2024-09-14T06:56:54.309 INFO:teuthology.orchestra.run.smithi007.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-09-14T06:56:54.334 INFO:teuthology.orchestra.run.smithi007.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.9 MB/s | 358 kB 00:00 2024-09-14T06:56:54.360 INFO:teuthology.orchestra.run.smithi007.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2024-09-14T06:56:54.385 INFO:teuthology.orchestra.run.smithi007.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-14T06:56:54.410 INFO:teuthology.orchestra.run.smithi007.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-14T06:56:54.436 INFO:teuthology.orchestra.run.smithi007.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-14T06:56:54.469 INFO:teuthology.orchestra.run.smithi007.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-14T06:56:54.494 INFO:teuthology.orchestra.run.smithi007.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-14T06:56:54.523 INFO:teuthology.orchestra.run.smithi007.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.0 MB/s | 230 kB 00:00 2024-09-14T06:56:54.548 INFO:teuthology.orchestra.run.smithi007.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.7 MB/s | 427 kB 00:00 2024-09-14T06:56:54.553 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:56:54.554 INFO:teuthology.orchestra.run.smithi007.stdout:Total 8.5 MB/s | 31 MB 00:03 2024-09-14T06:56:54.714 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:56:54.757 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:56:54.757 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:56:55.265 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:56:55.265 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:56:55.728 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T06:56:55.754 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T06:56:55.755 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:56:55.755 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T06:56:55.755 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T06:56:55.755 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T06:56:55.755 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:56.220 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T06:56:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T06:56:56.244 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:56:56.244 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T06:56:56.244 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T06:56:56.244 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T06:56:56.244 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:56.479 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T06:56:56.503 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T06:56:56.503 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:56:56.503 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T06:56:56.503 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T06:56:56.504 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T06:56:56.504 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:56.567 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T06:56:56.686 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:56:56.751 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T06:56:56.785 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T06:56:56.821 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T06:56:56.915 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T06:56:56.971 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T06:56:57.003 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T06:56:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T06:56:57.178 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T06:56:57.224 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T06:56:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T06:56:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T06:56:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T06:56:57.372 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T06:56:57.414 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T06:56:57.457 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T06:56:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T06:56:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T06:56:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T06:56:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T06:56:57.826 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T06:56:57.918 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T06:56:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T06:56:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T06:56:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T06:56:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T06:56:58.327 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T06:56:58.327 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 1/37 2024-09-14T06:56:58.327 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2/37 2024-09-14T06:56:58.327 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 3/37 2024-09-14T06:56:58.327 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 4/37 2024-09-14T06:56:58.327 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 5/37 2024-09-14T06:56:58.327 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 6/37 2024-09-14T06:56:58.327 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T06:56:58.327 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T06:56:58.328 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T06:56:58.329 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T06:56:58.329 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T06:56:58.329 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T06:56:58.329 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T06:56:58.329 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T06:56:58.329 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T06:56:58.330 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T06:56:58.330 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T06:56:58.330 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T06:56:58.330 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T06:56:58.330 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T06:56:58.330 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T06:56:58.330 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T06:56:58.330 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T06:56:58.388 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T06:56:58.440 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T06:56:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T06:56:58.628 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T06:56:58.672 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T06:56:58.817 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T06:56:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 32/37 2024-09-14T06:56:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T06:56:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T06:56:59.055 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:56:59.055 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T06:56:59.055 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T06:56:59.055 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T06:56:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:56:59.246 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T06:56:59.246 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:59.246 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:56:59.246 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:59.246 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:59.246 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T06:56:59.247 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T06:56:59.248 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T06:56:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T06:56:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T06:56:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T06:56:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T06:56:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T06:56:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T06:56:59.249 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:56:59.249 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:56:59.651 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-base 2024-09-14T06:57:00.228 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:52 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:00.319 INFO:teuthology.orchestra.run.smithi189.stdout:Package ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:00.361 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:00.362 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-14T06:57:00.362 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:57:00.438 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install cephadm 2024-09-14T06:57:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T06:57:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T06:57:00.638 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:57:00.638 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T06:57:00.638 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T06:57:00.638 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T06:57:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:01.003 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:53 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:01.117 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T06:57:01.137 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:01.138 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:01.138 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:57:01.138 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:01.138 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:57:01.138 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 783 k 2024-09-14T06:57:01.138 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:01.139 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:57:01.139 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:01.139 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-14T06:57:01.139 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:01.139 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 783 k 2024-09-14T06:57:01.139 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 789 k 2024-09-14T06:57:01.139 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:57:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T06:57:01.143 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:57:01.143 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T06:57:01.144 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T06:57:01.144 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T06:57:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:01.385 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T06:57:01.410 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T06:57:01.410 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:57:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T06:57:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T06:57:01.411 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T06:57:01.411 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:01.411 INFO:teuthology.orchestra.run.smithi189.stdout:cephadm-19.3.0-4775.g7f24e589.el9.noarch.rpm 2.8 MB/s | 783 kB 00:00 2024-09-14T06:57:01.412 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:01.412 INFO:teuthology.orchestra.run.smithi189.stdout:Total 2.8 MB/s | 783 kB 00:00 2024-09-14T06:57:01.412 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:57:01.417 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:57:01.417 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:57:01.423 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:57:01.424 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:57:01.477 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T06:57:01.499 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:57:01.943 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:02.033 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:02.383 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:02.630 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:57:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:02.630 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:02.630 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:57:02.817 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T06:57:03.285 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T06:57:03.285 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 1/37 2024-09-14T06:57:03.285 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2/37 2024-09-14T06:57:03.285 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 3/37 2024-09-14T06:57:03.285 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 4/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 5/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 6/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T06:57:03.286 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T06:57:03.287 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T06:57:03.288 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T06:57:03.288 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T06:57:03.288 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T06:57:03.289 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T06:57:03.289 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T06:57:03.289 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T06:57:03.289 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T06:57:03.289 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T06:57:03.399 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:55 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:03.535 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:03.535 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 145 k 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 145 k 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 432 k 2024-09-14T06:57:03.536 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:57:03.743 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-immutable-object-cache-19.3.0-4775.g7f24e5 702 kB/s | 145 kB 00:00 2024-09-14T06:57:03.744 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:03.744 INFO:teuthology.orchestra.run.smithi189.stdout:Total 696 kB/s | 145 kB 00:00 2024-09-14T06:57:03.744 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:57:03.752 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:57:03.752 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:57:03.810 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:57:03.810 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:57:03.965 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:57:04.036 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:57:04.061 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:57:04.061 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:57:04.061 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T06:57:04.061 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T06:57:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T06:57:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T06:57:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T06:57:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T06:57:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T06:57:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T06:57:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T06:57:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:04.251 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:04.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-base 2024-09-14T06:57:04.996 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:57:04.996 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:04.996 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:57:04.996 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:57:04.996 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:04.996 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:57:05.213 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr 2024-09-14T06:57:05.241 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:56 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:05.335 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:05.381 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:05.382 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-14T06:57:05.382 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:05.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install cephadm 2024-09-14T06:57:05.791 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:57 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:05.885 INFO:teuthology.orchestra.run.smithi189.stdout:Package ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:05.926 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:05.928 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-14T06:57:05.928 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:57:06.001 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T06:57:06.036 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:57 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 783 k 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 783 k 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 789 k 2024-09-14T06:57:06.177 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:06.434 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm-19.3.0-4775.g7f24e589.el9.noarch.rpm 3.0 MB/s | 783 kB 00:00 2024-09-14T06:57:06.434 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:06.434 INFO:teuthology.orchestra.run.smithi007.stdout:Total 3.0 MB/s | 783 kB 00:00 2024-09-14T06:57:06.435 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:06.440 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:06.440 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:06.447 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:06.447 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:06.571 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:00:58 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:06.704 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:06.705 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:06.705 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:57:06.705 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:06.705 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:57:06.705 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 14 M 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 34 k 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 16 k 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T06:57:06.706 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T06:57:06.707 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:06.707 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:57:06.707 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:06.707 INFO:teuthology.orchestra.run.smithi189.stdout:Install 12 Packages 2024-09-14T06:57:06.707 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:06.707 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 19 M 2024-09-14T06:57:06.708 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 159 M 2024-09-14T06:57:06.708 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:57:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:07.091 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:07.305 INFO:teuthology.orchestra.run.smithi189.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4775.g7f 240 kB/s | 34 kB 00:00 2024-09-14T06:57:07.330 INFO:teuthology.orchestra.run.smithi189.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4775.g7f2 98 kB/s | 16 kB 00:00 2024-09-14T06:57:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:07.472 INFO:teuthology.orchestra.run.smithi189.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 269 kB 00:00 2024-09-14T06:57:07.522 INFO:teuthology.orchestra.run.smithi189.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-09-14T06:57:07.572 INFO:teuthology.orchestra.run.smithi189.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-14T06:57:07.598 INFO:teuthology.orchestra.run.smithi189.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-14T06:57:07.665 INFO:teuthology.orchestra.run.smithi189.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-14T06:57:07.690 INFO:teuthology.orchestra.run.smithi189.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-14T06:57:07.714 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:07.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:07.714 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:07.714 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:07.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:07.716 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:07.716 INFO:teuthology.orchestra.run.smithi189.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-14T06:57:07.740 INFO:teuthology.orchestra.run.smithi189.stdout:(10/12): protobuf-compiler-3.14.0-13.el9.x86_64 3.1 MB/s | 863 kB 00:00 2024-09-14T06:57:07.765 INFO:teuthology.orchestra.run.smithi189.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-14T06:57:07.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T06:57:07.916 INFO:teuthology.orchestra.run.smithi189.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4775.g7f24e5 19 MB/s | 14 MB 00:00 2024-09-14T06:57:07.919 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:07.919 INFO:teuthology.orchestra.run.smithi189.stdout:Total 16 MB/s | 19 MB 00:01 2024-09-14T06:57:07.967 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:57:07.986 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:57:07.986 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:57:08.262 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:57:08.263 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:57:08.501 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:59 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:08.640 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:08.641 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:08.642 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-14T06:57:08.642 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:08.642 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:08.642 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 145 k 2024-09-14T06:57:08.643 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:08.643 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:08.643 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:08.643 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-14T06:57:08.643 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:08.643 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 145 k 2024-09-14T06:57:08.643 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 432 k 2024-09-14T06:57:08.643 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:08.823 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-immutable-object-cache-19.3.0-4775.g7f24e5 799 kB/s | 145 kB 00:00 2024-09-14T06:57:08.824 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:08.825 INFO:teuthology.orchestra.run.smithi007.stdout:Total 791 kB/s | 145 kB 00:00 2024-09-14T06:57:08.825 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:08.833 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:08.833 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:08.851 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:57:08.894 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:08.894 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T06:57:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T06:57:09.023 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T06:57:09.048 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:09.054 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T06:57:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:57:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:57:09.144 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:57:09.144 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T06:57:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:09.151 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T06:57:09.280 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T06:57:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T06:57:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T06:57:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T06:57:09.477 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 10/12 2024-09-14T06:57:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:57:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:10.076 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:57:10.076 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:10.076 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:10.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr 2024-09-14T06:57:10.890 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:01 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:10.984 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:11.029 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-14T06:57:11.030 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:11.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T06:57:11.688 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:02 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:11.824 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:11.825 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:11.825 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 14 M 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 34 k 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 16 k 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T06:57:11.826 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T06:57:11.827 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T06:57:11.827 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T06:57:11.827 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:11.827 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:11.827 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:11.827 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-09-14T06:57:11.827 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:11.827 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 19 M 2024-09-14T06:57:11.828 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 159 M 2024-09-14T06:57:11.828 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:12.419 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): ceph-prometheus-alerts-19.3.0-4775.g7f2 113 kB/s | 16 kB 00:00 2024-09-14T06:57:12.444 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): ceph-grafana-dashboards-19.3.0-4775.g7f 200 kB/s | 34 kB 00:00 2024-09-14T06:57:12.586 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 269 kB 00:00 2024-09-14T06:57:12.645 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-09-14T06:57:12.659 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 11/12 2024-09-14T06:57:12.712 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 8.0 MB/s | 548 kB 00:00 2024-09-14T06:57:12.737 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-14T06:57:12.748 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T06:57:12.804 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-14T06:57:12.829 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-14T06:57:12.855 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-14T06:57:12.880 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-14T06:57:12.905 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.6 MB/s | 863 kB 00:00 2024-09-14T06:57:13.064 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4775.g7f24e5 18 MB/s | 14 MB 00:00 2024-09-14T06:57:13.067 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:13.067 INFO:teuthology.orchestra.run.smithi007.stdout:Total 16 MB/s | 19 MB 00:01 2024-09-14T06:57:13.117 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:13.138 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:13.138 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:13.428 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:13.428 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:14.032 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:14.124 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T06:57:14.158 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T06:57:14.206 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T06:57:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T06:57:14.337 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T06:57:14.476 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T06:57:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T06:57:14.616 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T06:57:14.652 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T06:57:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T06:57:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 1/12 2024-09-14T06:57:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 2/12 2024-09-14T06:57:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 3/12 2024-09-14T06:57:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T06:57:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T06:57:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T06:57:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T06:57:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T06:57:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T06:57:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T06:57:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T06:57:14.685 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 10/12 2024-09-14T06:57:14.979 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T06:57:14.979 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:14.979 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:57:14.979 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:14.980 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:57:15.245 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T06:57:15.811 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:07 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:15.946 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 7.4 M 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T06:57:15.947 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout:======================================================================================= 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout:Install 12 Packages 2024-09-14T06:57:15.948 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:15.949 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 38 M 2024-09-14T06:57:15.949 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 212 M 2024-09-14T06:57:15.949 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:57:16.238 INFO:teuthology.orchestra.run.smithi189.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-14T06:57:16.280 INFO:teuthology.orchestra.run.smithi189.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-14T06:57:16.305 INFO:teuthology.orchestra.run.smithi189.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 490 kB/s | 33 kB 00:00 2024-09-14T06:57:16.347 INFO:teuthology.orchestra.run.smithi189.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 438 kB/s | 18 kB 00:00 2024-09-14T06:57:16.397 INFO:teuthology.orchestra.run.smithi189.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-09-14T06:57:16.422 INFO:teuthology.orchestra.run.smithi189.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-14T06:57:16.523 INFO:teuthology.orchestra.run.smithi189.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 39 MB/s | 4.9 MB 00:00 2024-09-14T06:57:16.548 INFO:teuthology.orchestra.run.smithi189.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-14T06:57:16.648 INFO:teuthology.orchestra.run.smithi189.stdout:(9/12): ceph-mgr-diskprediction-local-19.3.0-47 13 MB/s | 7.4 MB 00:00 2024-09-14T06:57:16.673 INFO:teuthology.orchestra.run.smithi189.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-09-14T06:57:16.749 INFO:teuthology.orchestra.run.smithi189.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-14T06:57:17.091 INFO:teuthology.orchestra.run.smithi189.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-14T06:57:17.091 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:17.092 INFO:teuthology.orchestra.run.smithi189.stdout:Total 33 MB/s | 38 MB 00:01 2024-09-14T06:57:17.449 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:57:17.475 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:57:17.475 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:57:17.909 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:57:17.910 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:57:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 11/12 2024-09-14T06:57:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T06:57:18.680 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:57:18.772 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T06:57:18.814 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T06:57:18.914 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T06:57:18.948 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T06:57:19.134 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T06:57:19.172 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T06:57:19.249 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T06:57:19.922 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T06:57:20.044 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 1/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 2/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 3/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T06:57:20.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T06:57:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T06:57:20.104 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T06:57:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T06:57:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:20.430 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T06:57:20.430 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:20.431 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:20.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T06:57:21.281 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:12 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:21.288 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T06:57:21.421 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:21.422 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-09-14T06:57:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:57:21.422 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-09-14T06:57:21.422 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 7.4 M 2024-09-14T06:57:21.422 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-14T06:57:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T06:57:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-09-14T06:57:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:21.424 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 38 M 2024-09-14T06:57:21.424 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 212 M 2024-09-14T06:57:21.424 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:21.831 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-14T06:57:21.872 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-14T06:57:21.923 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-14T06:57:21.965 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 437 kB/s | 18 kB 00:00 2024-09-14T06:57:22.007 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 927 kB/s | 38 kB 00:00 2024-09-14T06:57:22.124 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-47 16 MB/s | 7.4 MB 00:00 2024-09-14T06:57:22.182 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.7 MB/s | 3.0 MB 00:00 2024-09-14T06:57:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T06:57:22.332 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-09-14T06:57:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T06:57:22.408 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 12 MB/s | 4.9 MB 00:00 2024-09-14T06:57:22.433 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 792 kB/s | 245 kB 00:00 2024-09-14T06:57:22.458 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-09-14T06:57:23.025 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-14T06:57:23.026 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:23.026 INFO:teuthology.orchestra.run.smithi007.stdout:Total 24 MB/s | 38 MB 00:01 2024-09-14T06:57:23.393 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:23.421 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:23.421 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:23.888 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:23.888 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:24.681 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:24.757 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T06:57:24.799 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T06:57:24.904 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T06:57:24.942 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T06:57:24.968 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T06:57:24.968 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 1/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T06:57:24.969 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T06:57:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T06:57:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T06:57:25.253 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T06:57:25.395 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T06:57:25.396 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T06:57:25.397 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T06:57:25.397 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T06:57:25.397 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:25.397 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:57:25.767 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-rook 2024-09-14T06:57:25.946 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T06:57:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T06:57:26.346 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:18 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:26.480 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:26.481 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:26.481 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:57:26.481 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:26.481 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:57:26.481 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 49 k 2024-09-14T06:57:26.481 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-14T06:57:26.481 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T06:57:26.481 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:26.482 INFO:teuthology.orchestra.run.smithi189.stdout:Install 13 Packages 2024-09-14T06:57:26.483 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:26.483 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 2.2 M 2024-09-14T06:57:26.483 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 26 M 2024-09-14T06:57:26.483 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:57:27.154 INFO:teuthology.orchestra.run.smithi189.stdout:(1/13): ceph-mgr-rook-19.3.0-4775.g7f24e589.el9 272 kB/s | 49 kB 00:00 2024-09-14T06:57:27.238 INFO:teuthology.orchestra.run.smithi189.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 72 kB/s | 19 kB 00:00 2024-09-14T06:57:27.263 INFO:teuthology.orchestra.run.smithi189.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 88 kB/s | 26 kB 00:00 2024-09-14T06:57:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T06:57:27.663 INFO:teuthology.orchestra.run.smithi189.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 437 kB/s | 222 kB 00:00 2024-09-14T06:57:27.688 INFO:teuthology.orchestra.run.smithi189.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 352 kB/s | 159 kB 00:00 2024-09-14T06:57:27.730 INFO:teuthology.orchestra.run.smithi189.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 773 kB/s | 32 kB 00:00 2024-09-14T06:57:27.756 INFO:teuthology.orchestra.run.smithi189.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 567 kB/s | 14 kB 00:00 2024-09-14T06:57:27.781 INFO:teuthology.orchestra.run.smithi189.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 539 kB/s | 279 kB 00:00 2024-09-14T06:57:27.814 INFO:teuthology.orchestra.run.smithi189.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.7 MB/s | 220 kB 00:00 2024-09-14T06:57:27.840 INFO:teuthology.orchestra.run.smithi189.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2024-09-14T06:57:27.865 INFO:teuthology.orchestra.run.smithi189.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-14T06:57:27.907 INFO:teuthology.orchestra.run.smithi189.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.1 MB/s | 1.0 MB 00:00 2024-09-14T06:57:28.015 INFO:teuthology.orchestra.run.smithi189.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 153 kB/s | 54 kB 00:00 2024-09-14T06:57:28.018 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:28.019 INFO:teuthology.orchestra.run.smithi189.stdout:Total 1.4 MB/s | 2.2 MB 00:01 2024-09-14T06:57:28.089 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:57:28.105 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:57:28.105 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:57:28.193 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:57:28.194 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:57:28.455 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T06:57:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T06:57:28.703 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:57:28.869 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T06:57:28.924 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T06:57:28.981 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T06:57:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T06:57:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T06:57:29.170 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T06:57:29.278 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T06:57:29.335 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T06:57:30.053 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T06:57:30.116 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T06:57:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T06:57:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T06:57:30.227 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T06:57:31.005 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T06:57:31.005 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 1/13 2024-09-14T06:57:31.005 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T06:57:31.005 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T06:57:31.005 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T06:57:31.005 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T06:57:31.005 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T06:57:31.005 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T06:57:31.006 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T06:57:31.006 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T06:57:31.006 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T06:57:31.006 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T06:57:31.006 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T06:57:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T06:57:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 1/12 2024-09-14T06:57:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T06:57:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T06:57:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T06:57:31.326 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T06:57:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T06:57:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T06:57:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T06:57:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T06:57:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T06:57:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T06:57:31.333 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T06:57:31.333 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:31.333 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:57:31.333 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:31.333 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T06:57:31.333 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T06:57:31.333 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T06:57:31.333 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T06:57:31.333 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T06:57:31.334 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T06:57:31.334 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T06:57:31.334 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T06:57:31.334 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T06:57:31.334 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T06:57:31.334 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T06:57:31.334 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T06:57:31.334 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:31.334 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:57:31.605 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T06:57:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T06:57:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:31.744 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T06:57:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:31.746 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:32.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-rook 2024-09-14T06:57:32.176 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:24 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:32.310 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 154 k 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T06:57:32.311 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:32.312 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:57:32.312 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:32.312 INFO:teuthology.orchestra.run.smithi189.stdout:Install 6 Packages 2024-09-14T06:57:32.312 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:32.312 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 7.0 M 2024-09-14T06:57:32.312 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 33 M 2024-09-14T06:57:32.312 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:57:32.625 INFO:teuthology.orchestra.run.smithi189.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4775.g7f24e589.e 731 kB/s | 154 kB 00:00 2024-09-14T06:57:32.675 INFO:teuthology.orchestra.run.smithi189.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-14T06:57:32.700 INFO:teuthology.orchestra.run.smithi189.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-14T06:57:32.725 INFO:teuthology.orchestra.run.smithi189.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-14T06:57:32.727 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:23 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:32.869 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 49 k 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T06:57:32.871 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout:Install 13 Packages 2024-09-14T06:57:32.872 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:32.873 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 2.2 M 2024-09-14T06:57:32.873 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 26 M 2024-09-14T06:57:32.873 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:33.051 INFO:teuthology.orchestra.run.smithi189.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 390 kB/s | 248 kB 00:00 2024-09-14T06:57:33.249 INFO:teuthology.orchestra.run.smithi007.stdout:(1/13): ceph-mgr-rook-19.3.0-4775.g7f24e589.el9 319 kB/s | 49 kB 00:00 2024-09-14T06:57:33.591 INFO:teuthology.orchestra.run.smithi007.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 39 kB/s | 19 kB 00:00 2024-09-14T06:57:33.649 INFO:teuthology.orchestra.run.smithi007.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 46 kB/s | 26 kB 00:00 2024-09-14T06:57:33.991 INFO:teuthology.orchestra.run.smithi007.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 299 kB/s | 222 kB 00:00 2024-09-14T06:57:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 224 kB/s | 159 kB 00:00 2024-09-14T06:57:34.342 INFO:teuthology.orchestra.run.smithi007.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 796 kB/s | 32 kB 00:00 2024-09-14T06:57:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 567 kB/s | 14 kB 00:00 2024-09-14T06:57:34.392 INFO:teuthology.orchestra.run.smithi007.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.6 MB/s | 220 kB 00:00 2024-09-14T06:57:34.426 INFO:teuthology.orchestra.run.smithi007.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 359 kB/s | 279 kB 00:00 2024-09-14T06:57:34.468 INFO:teuthology.orchestra.run.smithi007.stdout:(10/13): python3-kubernetes-26.1.0-2.el9.noarch 14 MB/s | 1.0 MB 00:00 2024-09-14T06:57:34.493 INFO:teuthology.orchestra.run.smithi007.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-14T06:57:34.518 INFO:teuthology.orchestra.run.smithi007.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-09-14T06:57:34.635 INFO:teuthology.orchestra.run.smithi007.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 83 kB/s | 54 kB 00:00 2024-09-14T06:57:34.639 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:34.639 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.3 MB/s | 2.2 MB 00:01 2024-09-14T06:57:34.713 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:34.731 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:34.731 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:34.734 INFO:teuthology.orchestra.run.smithi189.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.6 MB/s | 6.0 MB 00:02 2024-09-14T06:57:34.736 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:34.736 INFO:teuthology.orchestra.run.smithi189.stdout:Total 2.9 MB/s | 7.0 MB 00:02 2024-09-14T06:57:34.809 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:57:34.821 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:57:34.821 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:57:34.833 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:34.834 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:34.920 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:57:34.920 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:57:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:57:35.375 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T06:57:35.419 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T06:57:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T06:57:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T06:57:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T06:57:35.838 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T06:57:35.880 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T06:57:35.944 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T06:57:35.995 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T06:57:36.030 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T06:57:36.098 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T06:57:36.105 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T06:57:36.123 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T06:57:36.158 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T06:57:36.904 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T06:57:36.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/6 2024-09-14T06:57:36.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T06:57:36.904 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T06:57:36.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T06:57:36.905 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T06:57:36.920 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T06:57:36.971 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T06:57:37.004 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T06:57:37.053 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T06:57:37.091 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T06:57:37.206 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T06:57:37.206 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:37.206 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:57:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T06:57:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T06:57:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T06:57:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T06:57:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T06:57:37.207 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:37.207 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:57:37.488 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-fuse 2024-09-14T06:57:37.836 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T06:57:37.836 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 1/13 2024-09-14T06:57:37.836 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T06:57:37.836 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T06:57:37.836 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T06:57:37.836 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T06:57:37.836 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T06:57:37.836 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T06:57:37.836 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T06:57:37.837 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T06:57:37.837 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T06:57:37.837 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T06:57:37.837 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T06:57:38.050 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:30 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:38.182 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:38.183 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:38.183 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:57:38.183 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:38.183 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:57:38.183 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 900 k 2024-09-14T06:57:38.183 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-09-14T06:57:38.184 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T06:57:38.184 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:38.184 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:57:38.184 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:38.184 INFO:teuthology.orchestra.run.smithi189.stdout:Install 2 Packages 2024-09-14T06:57:38.184 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:38.184 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 979 k 2024-09-14T06:57:38.184 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 2.9 M 2024-09-14T06:57:38.185 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:57:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T06:57:38.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T06:57:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T06:57:38.228 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:38.228 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:38.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T06:57:38.577 INFO:teuthology.orchestra.run.smithi189.stdout:(1/2): ceph-fuse-19.3.0-4775.g7f24e589.el9.x86_ 3.3 MB/s | 900 kB 00:00 2024-09-14T06:57:39.094 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:30 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:39.236 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:39.237 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:57:39.237 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:39.237 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 154 k 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout:Install 6 Packages 2024-09-14T06:57:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:39.239 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 7.0 M 2024-09-14T06:57:39.239 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 33 M 2024-09-14T06:57:39.239 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:39.838 INFO:teuthology.orchestra.run.smithi007.stdout:(1/6): ceph-mgr-cephadm-19.3.0-4775.g7f24e589.e 679 kB/s | 154 kB 00:00 2024-09-14T06:57:39.888 INFO:teuthology.orchestra.run.smithi007.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-14T06:57:39.905 INFO:teuthology.orchestra.run.smithi007.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-14T06:57:39.930 INFO:teuthology.orchestra.run.smithi007.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-14T06:57:40.247 INFO:teuthology.orchestra.run.smithi007.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 390 kB/s | 248 kB 00:00 2024-09-14T06:57:40.864 INFO:teuthology.orchestra.run.smithi007.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-09-14T06:57:40.866 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:40.867 INFO:teuthology.orchestra.run.smithi007.stdout:Total 4.3 MB/s | 7.0 MB 00:01 2024-09-14T06:57:40.943 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:40.955 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:40.956 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:41.061 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:41.063 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:41.495 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:41.670 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T06:57:41.726 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T06:57:42.285 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T06:57:42.382 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T06:57:42.456 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T06:57:42.487 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T06:57:43.261 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T06:57:43.261 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/6 2024-09-14T06:57:43.261 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T06:57:43.261 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T06:57:43.261 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T06:57:43.261 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T06:57:43.532 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:43.533 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:43.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-fuse 2024-09-14T06:57:44.394 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:35 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:44.533 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 900 k 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:44.534 INFO:teuthology.orchestra.run.smithi007.stdout:Install 2 Packages 2024-09-14T06:57:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:44.535 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 979 k 2024-09-14T06:57:44.535 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 2.9 M 2024-09-14T06:57:44.535 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:44.717 INFO:teuthology.orchestra.run.smithi007.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 635 kB/s | 80 kB 00:00 2024-09-14T06:57:44.858 INFO:teuthology.orchestra.run.smithi007.stdout:(2/2): ceph-fuse-19.3.0-4775.g7f24e589.el9.x86_ 3.3 MB/s | 900 kB 00:00 2024-09-14T06:57:44.859 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:44.859 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.9 MB/s | 979 kB 00:00 2024-09-14T06:57:44.871 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:44.894 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:44.894 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:44.979 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:44.980 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:45.324 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T06:57:45.395 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T06:57:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T06:57:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/2 2024-09-14T06:57:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T06:57:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:46.435 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T06:57:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:46.435 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:46.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-volume 2024-09-14T06:57:47.433 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:38 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:47.574 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:47.574 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 285 k 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 285 k 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 1.3 M 2024-09-14T06:57:47.575 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:47.805 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-volume-19.3.0-4775.g7f24e589.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-14T06:57:47.806 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:47.806 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-14T06:57:47.806 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:47.816 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:47.816 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:47.852 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:47.853 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:48.086 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:48.157 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:48.175 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:57:48.175 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T06:57:48.175 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:49.218 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:57:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:49.218 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:49.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados-devel 2024-09-14T06:57:50.121 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:41 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:50.265 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:50.265 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-14T06:57:50.265 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:50.265 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:50.265 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 126 k 2024-09-14T06:57:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:50.266 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:50.266 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:50.266 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-14T06:57:50.266 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:50.266 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 126 k 2024-09-14T06:57:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 456 k 2024-09-14T06:57:50.267 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:50.478 INFO:teuthology.orchestra.run.smithi007.stdout:librados-devel-19.3.0-4775.g7f24e589.el9.x86_64 599 kB/s | 126 kB 00:00 2024-09-14T06:57:50.479 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:50.479 INFO:teuthology.orchestra.run.smithi007.stdout:Total 592 kB/s | 126 kB 00:00 2024-09-14T06:57:50.480 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:50.486 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:50.486 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:50.555 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:50.555 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:50.718 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:50.821 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:57:51.300 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:57:51.555 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:57:51.556 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:51.556 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:51.556 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:57:51.556 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:51.556 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:51.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs2 2024-09-14T06:57:52.393 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:43 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:52.486 INFO:teuthology.orchestra.run.smithi007.stdout:Package libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:52.533 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:52.534 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-14T06:57:52.535 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:52.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs-devel 2024-09-14T06:57:53.194 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:44 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:53.336 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 32 k 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:53.337 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 32 k 2024-09-14T06:57:53.338 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 142 k 2024-09-14T06:57:53.338 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:57:53.499 INFO:teuthology.orchestra.run.smithi007.stdout:libcephfs-devel-19.3.0-4775.g7f24e589.el9.x86_6 197 kB/s | 32 kB 00:00 2024-09-14T06:57:53.500 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:53.500 INFO:teuthology.orchestra.run.smithi007.stdout:Total 195 kB/s | 32 kB 00:00 2024-09-14T06:57:53.500 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:57:53.503 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:57:53.503 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:57:53.518 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:57:53.519 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:57:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:57:53.724 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:57:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:57:54.397 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:57:54.397 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:54.397 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:57:54.397 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:57:54.397 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:57:54.397 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:54.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados2 2024-09-14T06:57:55.180 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:46 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:55.274 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:55.320 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:55.321 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-14T06:57:55.321 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:55.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librbd1 2024-09-14T06:57:55.975 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:46 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:56.050 INFO:teuthology.orchestra.run.smithi189.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 4.5 kB/s | 80 kB 00:17 2024-09-14T06:57:56.051 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:56.051 INFO:teuthology.orchestra.run.smithi189.stdout:Total 55 kB/s | 979 kB 00:17 2024-09-14T06:57:56.064 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:57:56.068 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:56.089 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:57:56.090 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:57:56.115 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:56.116 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-14T06:57:56.116 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:56.173 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:57:56.173 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:57:56.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rados 2024-09-14T06:57:56.360 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:57:56.502 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T06:57:56.565 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T06:57:56.772 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:47 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:56.865 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:56.910 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:56.911 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-14T06:57:56.911 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:56.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rgw 2024-09-14T06:57:57.268 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T06:57:57.268 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/2 2024-09-14T06:57:57.547 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T06:57:57.548 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:57.548 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:57:57.548 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T06:57:57.548 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:57.548 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:57:57.572 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:48 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:57.666 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:57.712 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:57.713 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-14T06:57:57.713 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:57.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-cephfs 2024-09-14T06:57:57.844 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-volume 2024-09-14T06:57:58.387 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:49 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:58.425 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:50 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:57:58.482 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:58.530 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:58.531 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-14T06:57:58.531 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:58.561 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:57:58.561 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:58.561 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 285 k 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 285 k 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 1.3 M 2024-09-14T06:57:58.562 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:57:58.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rbd 2024-09-14T06:57:58.796 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-volume-19.3.0-4775.g7f24e589.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-14T06:57:58.797 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:57:58.797 INFO:teuthology.orchestra.run.smithi189.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-14T06:57:58.797 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:57:58.805 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:57:58.805 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:57:58.839 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:57:58.839 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:57:59.064 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:57:59.130 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:59.146 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:57:59.146 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:57:59.147 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T06:57:59.147 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:57:59.188 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:50 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:57:59.282 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:57:59.328 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:57:59.329 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-14T06:57:59.329 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:57:59.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-fuse 2024-09-14T06:57:59.985 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:50 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:58:00.094 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:58:00.094 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:00.094 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:58:00.095 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:58:00.095 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:00.095 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:00.128 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:58:00.128 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:58:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-14T06:58:00.128 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 85 k 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 85 k 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 231 k 2024-09-14T06:58:00.129 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:58:00.327 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-fuse-19.3.0-4775.g7f24e589.el9.x86_64.rpm 423 kB/s | 85 kB 00:00 2024-09-14T06:58:00.328 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:58:00.328 INFO:teuthology.orchestra.run.smithi007.stdout:Total 420 kB/s | 85 kB 00:00 2024-09-14T06:58:00.328 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:58:00.336 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:58:00.336 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:58:00.384 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librados-devel 2024-09-14T06:58:00.403 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:58:00.403 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:58:00.551 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:58:00.670 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:00.960 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:52 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:01.061 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:01.101 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:01.101 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:01.101 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-14T06:58:01.101 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:01.102 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:58:01.102 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 126 k 2024-09-14T06:58:01.102 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:01.102 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:58:01.102 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:01.102 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-14T06:58:01.102 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:01.102 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 126 k 2024-09-14T06:58:01.103 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 456 k 2024-09-14T06:58:01.103 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:58:01.308 INFO:teuthology.orchestra.run.smithi189.stdout:librados-devel-19.3.0-4775.g7f24e589.el9.x86_64 616 kB/s | 126 kB 00:00 2024-09-14T06:58:01.309 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:58:01.309 INFO:teuthology.orchestra.run.smithi189.stdout:Total 611 kB/s | 126 kB 00:00 2024-09-14T06:58:01.309 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:58:01.314 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:58:01.315 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:58:01.321 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:01.321 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:01.321 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:58:01.321 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:58:01.321 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:01.321 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:58:01.379 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:58:01.379 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:58:01.533 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:58:01.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-mirror 2024-09-14T06:58:01.639 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:02.068 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:02.198 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:53 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:58:02.329 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:02.329 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:02.329 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:58:02.329 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:58:02.329 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:02.329 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:02.339 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:58:02.340 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:58:02.340 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 3.2 M 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 14 M 2024-09-14T06:58:02.341 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:58:02.603 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install libcephfs2 2024-09-14T06:58:02.698 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-mirror-19.3.0-4775.g7f24e589.el9.x86_64.rpm 8.9 MB/s | 3.2 MB 00:00 2024-09-14T06:58:02.698 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:58:02.699 INFO:teuthology.orchestra.run.smithi007.stdout:Total 8.9 MB/s | 3.2 MB 00:00 2024-09-14T06:58:02.699 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:58:02.710 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:58:02.710 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:58:02.797 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:58:02.798 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:58:03.185 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:55 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:03.261 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:58:03.277 INFO:teuthology.orchestra.run.smithi189.stdout:Package libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:58:03.322 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:03.323 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-14T06:58:03.323 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:03.367 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:58:03.368 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T06:58:03.368 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T06:58:03.368 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T06:58:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:03.392 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install libcephfs-devel 2024-09-14T06:58:03.962 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:55 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:04.098 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 32 k 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 32 k 2024-09-14T06:58:04.099 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 142 k 2024-09-14T06:58:04.100 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:58:04.244 INFO:teuthology.orchestra.run.smithi189.stdout:libcephfs-devel-19.3.0-4775.g7f24e589.el9.x86_6 219 kB/s | 32 kB 00:00 2024-09-14T06:58:04.245 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:58:04.245 INFO:teuthology.orchestra.run.smithi189.stdout:Total 217 kB/s | 32 kB 00:00 2024-09-14T06:58:04.245 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:58:04.249 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:58:04.249 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:58:04.269 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:58:04.269 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:58:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:58:04.450 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:04.521 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:04.521 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:58:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:58:04.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:04.522 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:58:04.829 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:04.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-nbd 2024-09-14T06:58:05.078 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:05.078 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:05.078 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:58:05.078 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:58:05.078 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:05.078 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:05.269 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librados2 2024-09-14T06:58:05.489 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:56 ago on Sat 14 Sep 2024 06:56:09 AM UTC. 2024-09-14T06:58:05.628 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-14T06:58:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:58:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-14T06:58:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:58:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-14T06:58:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 171 k 2024-09-14T06:58:05.629 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-14T06:58:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-14T06:58:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-14T06:58:05.630 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:05.630 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 171 k 2024-09-14T06:58:05.630 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 490 k 2024-09-14T06:58:05.630 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-14T06:58:05.828 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:57 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:05.843 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-nbd-19.3.0-4775.g7f24e589.el9.x86_64.rpm 801 kB/s | 171 kB 00:00 2024-09-14T06:58:05.843 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:58:05.844 INFO:teuthology.orchestra.run.smithi007.stdout:Total 796 kB/s | 171 kB 00:00 2024-09-14T06:58:05.844 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-14T06:58:05.852 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-14T06:58:05.852 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-14T06:58:05.919 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:58:05.920 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-14T06:58:05.920 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-14T06:58:05.963 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:05.963 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-14T06:58:05.964 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:06.059 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install librbd1 2024-09-14T06:58:06.103 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-14T06:58:06.210 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:06.621 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:58 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:06.637 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:06.713 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:58:06.756 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:06.757 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-14T06:58:06.757 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:06.833 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rados 2024-09-14T06:58:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:06.920 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-14T06:58:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:58:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:06.920 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-14T06:58:07.166 DEBUG:teuthology.parallel:result is None 2024-09-14T06:58:07.398 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:01:59 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:07.487 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:58:07.530 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:07.531 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-14T06:58:07.531 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:07.610 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rgw 2024-09-14T06:58:08.171 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:00 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:08.263 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:58:08.306 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:08.307 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-14T06:58:08.307 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:08.381 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-cephfs 2024-09-14T06:58:08.951 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:00 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:09.043 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:58:09.085 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:09.087 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-14T06:58:09.087 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:09.158 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install python3-rbd 2024-09-14T06:58:09.723 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:01 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:09.814 INFO:teuthology.orchestra.run.smithi189.stdout:Package python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:58:09.858 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:09.859 INFO:teuthology.orchestra.run.smithi189.stdout:Nothing to do. 2024-09-14T06:58:09.859 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:09.930 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-fuse 2024-09-14T06:58:10.490 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:02 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:10.624 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:10.625 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:10.625 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-14T06:58:10.625 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:10.625 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:58:10.625 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 85 k 2024-09-14T06:58:10.625 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:10.625 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:58:10.626 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:10.626 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-14T06:58:10.626 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:10.626 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 85 k 2024-09-14T06:58:10.626 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 231 k 2024-09-14T06:58:10.626 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:58:10.814 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-fuse-19.3.0-4775.g7f24e589.el9.x86_64.rpm 453 kB/s | 85 kB 00:00 2024-09-14T06:58:10.815 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:58:10.815 INFO:teuthology.orchestra.run.smithi189.stdout:Total 450 kB/s | 85 kB 00:00 2024-09-14T06:58:10.815 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:58:10.821 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:58:10.822 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:58:10.884 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:58:10.885 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:58:11.027 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:58:11.128 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:11.518 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:11.785 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:11.785 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:11.785 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:58:11.785 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:58:11.785 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:11.785 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:12.058 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-mirror 2024-09-14T06:58:12.639 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:04 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:12.777 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repo Size 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:12.778 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 3.2 M 2024-09-14T06:58:12.779 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 14 M 2024-09-14T06:58:12.779 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:58:13.108 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-mirror-19.3.0-4775.g7f24e589.el9.x86_64.rpm 9.6 MB/s | 3.2 MB 00:00 2024-09-14T06:58:13.109 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:58:13.109 INFO:teuthology.orchestra.run.smithi189.stdout:Total 9.6 MB/s | 3.2 MB 00:00 2024-09-14T06:58:13.110 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:58:13.119 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:58:13.119 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:58:13.200 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:58:13.201 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:58:13.636 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:58:13.701 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:13.725 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:13.725 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:58:13.725 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T06:58:13.726 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T06:58:13.726 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T06:58:13.726 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:14.808 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:14.809 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:14.809 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:58:14.809 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:58:14.809 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:14.809 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:15.036 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install rbd-nbd 2024-09-14T06:58:15.623 INFO:teuthology.orchestra.run.smithi189.stdout:Last metadata expiration check: 0:02:07 ago on Sat 14 Sep 2024 06:56:08 AM UTC. 2024-09-14T06:58:15.758 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-09-14T06:58:15.758 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout: Package Architecture Version Repository Size 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 171 k 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout:================================================================================ 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout:Install 1 Package 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 171 k 2024-09-14T06:58:15.759 INFO:teuthology.orchestra.run.smithi189.stdout:Installed size: 490 k 2024-09-14T06:58:15.760 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-09-14T06:58:15.969 INFO:teuthology.orchestra.run.smithi189.stdout:rbd-nbd-19.3.0-4775.g7f24e589.el9.x86_64.rpm 814 kB/s | 171 kB 00:00 2024-09-14T06:58:15.970 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:58:15.970 INFO:teuthology.orchestra.run.smithi189.stdout:Total 808 kB/s | 171 kB 00:00 2024-09-14T06:58:15.970 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-09-14T06:58:15.978 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-09-14T06:58:15.978 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-09-14T06:58:16.042 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-09-14T06:58:16.042 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-09-14T06:58:16.201 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-09-14T06:58:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:16.671 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-09-14T06:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:16.910 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-09-14T06:58:17.158 DEBUG:teuthology.parallel:result is None 2024-09-14T06:58:17.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:58:17.315 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T06:58:17.345 INFO:teuthology.orchestra.run.smithi007.stdout:19.3.0-4775.g7f24e589.el9 2024-09-14T06:58:17.345 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775.g7f24e589.el9 2024-09-14T06:58:17.346 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775.g7f24e589 is installed. 2024-09-14T06:58:17.347 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:58:17.515 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T06:58:17.537 INFO:teuthology.orchestra.run.smithi189.stdout:19.3.0-4775.g7f24e589.el9 2024-09-14T06:58:17.537 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775.g7f24e589.el9 2024-09-14T06:58:17.537 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775.g7f24e589 is installed. 2024-09-14T06:58:17.539 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-14T06:58:17.539 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:17.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T06:58:17.571 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:17.571 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T06:58:17.602 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-14T06:58:17.603 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:17.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T06:58:17.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T06:58:17.712 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:17.712 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T06:58:17.738 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T06:58:17.803 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-14T06:58:17.804 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:17.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T06:58:17.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T06:58:17.902 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:17.902 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T06:58:17.927 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T06:58:17.993 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-14T06:58:17.993 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:17.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T06:58:18.021 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T06:58:18.090 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:18.090 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T06:58:18.115 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T06:58:18.180 INFO:teuthology.run_tasks:Running task ceph... 2024-09-14T06:58:18.259 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-14T06:58:18.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-09-14T06:58:18.261 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /var/log/ceph 2024-09-14T06:58:18.286 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-14T06:58:18.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T06:58:18.328 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T06:58:18.353 INFO:tasks.ceph:Creating extra log directories... 2024-09-14T06:58:18.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T06:58:18.394 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T06:58:18.427 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-14T06:58:18.428 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-14T06:58:18.428 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408', 'branch': 'main', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900408', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5839, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'sleep_before_teardown': 0, 'subset': '99/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a', 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNiz1AK4+dA8lCcAIIAUsp7qbRBuuF0eIJrvNttXkv+D3S4YTAI0Fn2nUr+RGNdfM/aGH88B9MKTRUJ8AtkJkjM=', 'smithi189.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHks/XT6TB4R3tsjzw2PHsm9ZsH7tb0EA/5KHGn2x0bFxVEMLthTCe0TCq/zSVz5qbOPtEh6Ua7WVbmCtWxoaRE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_20:16:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-14T06:58:18.428 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T06:58:18.460 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T06:58:18.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T06:58:18.516 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T06:58:18.541 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:18.541 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T06:58:18.599 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T06:58:18.599 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-09-14T06:58:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T06:58:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:58:18.655 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-09-14T06:58:18.656 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:58:18.656 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:58:18.656 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-14 06:58:03.638021592 +0000 2024-09-14T06:58:18.656 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-14 06:55:32.659300651 +0000 2024-09-14T06:58:18.656 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-14 06:55:32.659300651 +0000 2024-09-14T06:58:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-14 06:55:32.659300651 +0000 2024-09-14T06:58:18.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T06:58:18.720 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-14T06:58:18.720 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-14T06:58:18.720 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 8.3817e-05 s, 6.1 MB/s 2024-09-14T06:58:18.721 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T06:58:18.779 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-09-14T06:58:18.836 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T06:58:18.836 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:58:18.836 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-09-14T06:58:18.836 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:58:18.836 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:58:18.836 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-14 06:58:03.638021592 +0000 2024-09-14T06:58:18.836 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-14 06:55:32.975296361 +0000 2024-09-14T06:58:18.836 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-14 06:55:32.975296361 +0000 2024-09-14T06:58:18.836 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-14 06:55:32.975296361 +0000 2024-09-14T06:58:18.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T06:58:18.900 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-14T06:58:18.900 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-14T06:58:18.900 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000102276 s, 5.0 MB/s 2024-09-14T06:58:18.901 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T06:58:18.959 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-09-14T06:58:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T06:58:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:58:19.015 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 707 Links: 1 2024-09-14T06:58:19.015 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:58:19.015 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:58:19.015 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-14 06:58:03.639021578 +0000 2024-09-14T06:58:19.015 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-14 06:55:33.285292154 +0000 2024-09-14T06:58:19.015 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-14 06:55:33.285292154 +0000 2024-09-14T06:58:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-14 06:55:33.285292154 +0000 2024-09-14T06:58:19.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T06:58:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-14T06:58:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-14T06:58:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000101557 s, 5.0 MB/s 2024-09-14T06:58:19.080 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T06:58:19.136 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-09-14T06:58:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T06:58:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:58:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 727 Links: 1 2024-09-14T06:58:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:58:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:58:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-14 06:58:03.639021578 +0000 2024-09-14T06:58:19.192 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-14 06:55:33.616287661 +0000 2024-09-14T06:58:19.193 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-14 06:55:33.616287661 +0000 2024-09-14T06:58:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-14 06:55:33.616287661 +0000 2024-09-14T06:58:19.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T06:58:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-14T06:58:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-14T06:58:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000100968 s, 5.1 MB/s 2024-09-14T06:58:19.258 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T06:58:19.314 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:19.314 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:19.314 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T06:58:19.333 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T06:58:19.333 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-09-14T06:58:19.390 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T06:58:19.390 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:58:19.390 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-14T06:58:19.390 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:58:19.390 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:58:19.390 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-14 06:58:13.988222032 +0000 2024-09-14T06:58:19.390 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-14 06:55:28.498295167 +0000 2024-09-14T06:58:19.390 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-14 06:55:28.498295167 +0000 2024-09-14T06:58:19.391 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-14 06:55:28.498295167 +0000 2024-09-14T06:58:19.391 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T06:58:19.456 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-14T06:58:19.456 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-14T06:58:19.456 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000231531 s, 2.2 MB/s 2024-09-14T06:58:19.457 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T06:58:19.515 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-09-14T06:58:19.572 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T06:58:19.572 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:58:19.572 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-14T06:58:19.572 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:58:19.572 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:58:19.572 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-14 06:58:13.988222032 +0000 2024-09-14T06:58:19.572 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-14 06:55:28.826288800 +0000 2024-09-14T06:58:19.572 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-14 06:55:28.826288800 +0000 2024-09-14T06:58:19.573 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-14 06:55:28.826288800 +0000 2024-09-14T06:58:19.573 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T06:58:19.636 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-14T06:58:19.636 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-14T06:58:19.636 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000215962 s, 2.4 MB/s 2024-09-14T06:58:19.637 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T06:58:19.695 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-09-14T06:58:19.750 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T06:58:19.750 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:58:19.750 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-14T06:58:19.751 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:58:19.751 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:58:19.751 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-14 06:58:13.988222032 +0000 2024-09-14T06:58:19.751 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-14 06:55:29.146282588 +0000 2024-09-14T06:58:19.751 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-14 06:55:29.146282588 +0000 2024-09-14T06:58:19.751 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-14 06:55:29.146282588 +0000 2024-09-14T06:58:19.751 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T06:58:19.814 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-14T06:58:19.814 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-14T06:58:19.814 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000200312 s, 2.6 MB/s 2024-09-14T06:58:19.815 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T06:58:19.872 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-09-14T06:58:19.927 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T06:58:19.928 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:58:19.928 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-14T06:58:19.928 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:58:19.928 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:58:19.928 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-14 06:58:13.989222013 +0000 2024-09-14T06:58:19.928 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-14 06:55:29.481276085 +0000 2024-09-14T06:58:19.928 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-14 06:55:29.481276085 +0000 2024-09-14T06:58:19.928 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-09-14 06:55:29.481276085 +0000 2024-09-14T06:58:19.929 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T06:58:19.991 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-14T06:58:19.991 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-14T06:58:19.991 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000237529 s, 2.2 MB/s 2024-09-14T06:58:19.992 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T06:58:20.049 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:20.049 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi189.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T06:58:20.049 INFO:tasks.ceph:Generating config... 2024-09-14T06:58:20.050 INFO:tasks.ceph:[client] rbd cache = False 2024-09-14T06:58:20.050 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-14T06:58:20.051 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-14T06:58:20.051 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-14T06:58:20.051 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-14T06:58:20.051 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-14T06:58:20.051 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-14T06:58:20.051 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-14T06:58:20.051 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-14T06:58:20.051 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-14T06:58:20.051 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-14T06:58:20.051 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-14T06:58:20.051 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-14T06:58:20.051 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-14T06:58:20.051 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-14T06:58:20.051 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-14T06:58:20.052 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-14T06:58:20.052 INFO:tasks.ceph:Setting up mon.a... 2024-09-14T06:58:20.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-14T06:58:20.111 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-09-14T06:58:20.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-14T06:58:20.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T06:58:20.188 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.c', '[v2:172.21.15.7:3301,v1:172.21.15.7:6790]'), ('mon.b', '172.21.15.189')] 2024-09-14T06:58:20.189 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.189', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': False}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-14T06:58:20.190 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:20.190 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T06:58:20.245 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --addv c '[v2:172.21.15.7:3301,v1:172.21.15.7:6790]' --add b 172.21.15.189 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-09-14T06:58:20.335008+0000 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-14T06:58:20.335008+0000 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 17 (quincy) 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-14T06:58:20.335 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-14T06:58:20.336 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] mon.c 2024-09-14T06:58:20.336 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-14T06:58:20.337 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T06:58:20.352 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6... 2024-09-14T06:58:20.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T06:58:20.395 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-14T06:58:20.438 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-14T06:58:20.439 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-14T06:58:20.440 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.189" 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: mon client directed command retry = 5 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on pool no app = False 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject socket failures = 5000 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-14T06:58:20.441 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.442 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore compression algorithm = zstd 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore compression mode = aggressive 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-14T06:58:20.443 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: osd blocked scrub grace period = 3600 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: osd max markdown count = 1000 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: osd shutdown pgref assert = True 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.444 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-14T06:58:20.445 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T06:58:20.446 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T06:58:20.447 INFO:teuthology.orchestra.run.smithi007.stdout: rbd cache = False 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi189.stdout:[global] 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: chdir = "" 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: auth supported = cephx 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: filestore xattr use omap = true 2024-09-14T06:58:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: mon clock drift allowed = 1.000 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: osd crush chooseleaf type = 0 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: auth debug = true 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on old message = true 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on bug = true 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: mon pg warn max object skew = 0 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default size = 2 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow primary affinity = true 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow pg remap = true 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on legacy crush tunables = false 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on crush straw calc version zero = false 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on no sortbitwise = false 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on osd down out interval zero = false 2024-09-14T06:58:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on too few osds = false 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: mon_allow_pool_size_one = true 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: osd default data pool replay window = 5 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: mon allow pool delete = true 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: mon cluster log file level = debug 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: debug asserts on shutdown = true 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: mon health detail to clog = false 2024-09-14T06:58:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: mon host = "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.189" 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: mon client directed command retry = 5 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on pool no app = False 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: ms inject socket failures = 5000 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: fsid = 32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout:[osd] 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: osd journal size = 100 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub load threshold = 5.0 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub max interval = 600 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: osd mclock profile = high_recovery_ops 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: osd recover clone overlap = true 2024-09-14T06:58:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: osd recovery max chunk = 1048576 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug shutdown = true 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug op order = true 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify stray on activate = true 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug trim objects = true 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: osd open classes on start = true 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug pg log writeout = true 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: osd deep scrub update digest min age = 30 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: osd map max advance = 10 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: journal zero on create = true 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: filestore ondisk finisher threads = 3 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: filestore apply finisher threads = 3 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: bdev debug aio = true 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug misdirected ops = true 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore block size = 96636764160 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore compression algorithm = zstd 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore compression mode = aggressive 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore fsck on mount = True 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluefs = 1/20 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluestore = 1/20 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: debug osd = 20 2024-09-14T06:58:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: debug rocksdb = 4/10 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd full ratio = 0.9 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd nearfull ratio = 0.8 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: osd blocked scrub grace period = 3600 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: osd failsafe full ratio = 0.95 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: osd max markdown count = 1000 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: osd objectstore = bluestore 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: osd shutdown pgref assert = True 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr] 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: debug mgr = 20 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-14T06:58:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: mgr/telemetry/nag = false 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout:[mon] 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: debug paxos = 20 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: mon data avail warn = 5 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: mon mgr mkfs grace = 240 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd reporter subtree level = osd 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd prime pg temp = true 2024-09-14T06:58:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: # 1m isn't quite enough 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: mon_down_mkfs_grace = 2m 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_filestore_osds = false 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-14T06:58:20.457 INFO:teuthology.orchestra.run.smithi189.stdout:[client] 2024-09-14T06:58:20.458 INFO:teuthology.orchestra.run.smithi189.stdout: rgw cache enabled = true 2024-09-14T06:58:20.458 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable ops log = true 2024-09-14T06:58:20.458 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable usage log = true 2024-09-14T06:58:20.458 INFO:teuthology.orchestra.run.smithi189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T06:58:20.458 INFO:teuthology.orchestra.run.smithi189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T06:58:20.458 INFO:teuthology.orchestra.run.smithi189.stdout: rbd cache = False 2024-09-14T06:58:20.458 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.a] 2024-09-14T06:58:20.458 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.c] 2024-09-14T06:58:20.458 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.b] 2024-09-14T06:58:20.459 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-14T06:58:20.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-14T06:58:20.556 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-14T06:58:20.556 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:20.556 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-14T06:58:20.572 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:20.572 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-14T06:58:20.628 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T06:58:20.628 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:20.628 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T06:58:20.628 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T06:58:20.704 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:20.704 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:20.760 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T06:58:20.760 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:20.760 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T06:58:20.760 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T06:58:20.796 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:20.796 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:20.853 INFO:tasks.ceph:Setting up mon nodes... 2024-09-14T06:58:20.853 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-14T06:58:20.853 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T06:58:20.906 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T06:58:20.908 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T06:58:20.973 INFO:teuthology.orchestra.run.smithi189.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T06:58:20.975 INFO:tasks.ceph:Setting up mds nodes... 2024-09-14T06:58:20.975 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-14T06:58:20.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-14T06:58:21.018 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-14T06:58:21.031 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-14T06:58:21.073 INFO:teuthology.orchestra.run.smithi189.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-14T06:58:21.086 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-14T06:58:21.086 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi189.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T06:58:21.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-14T06:58:21.111 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:21.111 INFO:tasks.ceph:role: osd.0 2024-09-14T06:58:21.111 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T06:58:21.112 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T06:58:21.177 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:58:21.177 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:58:21.177 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:58:21.178 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:58:21.178 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:58:21.178 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:58:21.178 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:58:21.178 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:58:21.178 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:58:21.178 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:58:21.209 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-14T06:58:21.211 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-14T06:58:21.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-14T06:58:21.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-14T06:58:21.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-14T06:58:21.418 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:21.418 INFO:tasks.ceph:role: osd.1 2024-09-14T06:58:21.418 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T06:58:21.419 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T06:58:21.483 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:58:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:58:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:58:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:58:21.484 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:58:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:58:21.484 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:58:21.484 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:58:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:58:21.484 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:58:21.516 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-14T06:58:21.518 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-14T06:58:21.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-14T06:58:21.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-14T06:58:21.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-14T06:58:21.726 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:21.726 INFO:tasks.ceph:role: osd.2 2024-09-14T06:58:21.726 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T06:58:21.726 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T06:58:21.790 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:58:21.791 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:58:21.791 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:58:21.791 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:58:21.791 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:58:21.791 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:58:21.791 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:58:21.791 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:58:21.791 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:58:21.791 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:58:21.823 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-14T06:58:21.825 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-14T06:58:21.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-14T06:58:21.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-14T06:58:21.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-14T06:58:22.028 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:22.028 INFO:tasks.ceph:role: osd.3 2024-09-14T06:58:22.028 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T06:58:22.028 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T06:58:22.094 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:58:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:58:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:58:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:58:22.094 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:58:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:58:22.094 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:58:22.094 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:58:22.094 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:58:22.095 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:58:22.126 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-14T06:58:22.128 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-14T06:58:22.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-14T06:58:22.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-14T06:58:22.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:22.367+0000 7f279b818640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-14T06:58:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:22.367+0000 7f279b818640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-14T06:58:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:22.367+0000 7f279b818640 -1 bdev(0x55ad35a0b800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-14T06:58:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:22.368+0000 7f279b818640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-14T06:58:25.779 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-14T06:58:25.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:25.902+0000 7f3d081ae640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-14T06:58:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:25.902+0000 7f3d081ae640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-14T06:58:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:25.902+0000 7f3d081ae640 -1 bdev(0x5620de939800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-14T06:58:25.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:25.902+0000 7f3d081ae640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-14T06:58:29.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-14T06:58:29.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:29.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:29.184+0000 7fcf35816640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-14T06:58:29.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:29.184+0000 7fcf35816640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-14T06:58:29.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:29.184+0000 7fcf35816640 -1 bdev(0x55adcb759800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-14T06:58:29.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:29.185+0000 7fcf35816640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-14T06:58:32.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-14T06:58:32.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:32.222+0000 7f44812da640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-14T06:58:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:32.223+0000 7f44812da640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-14T06:58:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:32.223+0000 7f44812da640 -1 bdev(0x555b3ea15800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-14T06:58:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-14T06:58:32.223+0000 7f44812da640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-14T06:58:35.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-14T06:58:35.417 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-14T06:58:35.448 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:35.448 INFO:tasks.ceph:role: osd.4 2024-09-14T06:58:35.448 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T06:58:35.448 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T06:58:35.523 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:58:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:58:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:58:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:58:35.524 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:58:35.524 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:58:35.524 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:58:35.524 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:58:35.525 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:58:35.525 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:58:36.035 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2024-09-14T06:58:36.037 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-14T06:58:36.038 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-14T06:58:36.187 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-14T06:58:36.221 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-14T06:58:36.288 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:36.289 INFO:tasks.ceph:role: osd.5 2024-09-14T06:58:36.289 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T06:58:36.289 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T06:58:36.362 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:58:36.362 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:58:36.362 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:58:36.362 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:58:36.363 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:58:36.363 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:58:36.363 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:58:36.363 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:58:36.363 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:58:36.363 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:58:36.796 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2024-09-14T06:58:36.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-14T06:58:36.799 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-14T06:58:36.890 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-14T06:58:36.919 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-14T06:58:36.983 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:36.984 INFO:tasks.ceph:role: osd.6 2024-09-14T06:58:36.984 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T06:58:36.984 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T06:58:37.056 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:58:37.057 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:58:37.057 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:58:37.057 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:58:37.057 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:58:37.057 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:58:37.057 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:58:37.057 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:58:37.057 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:58:37.057 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:58:37.546 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2024-09-14T06:58:37.548 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-14T06:58:37.548 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-14T06:58:37.701 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-14T06:58:37.733 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-14T06:58:37.800 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:58:37.800 INFO:tasks.ceph:role: osd.7 2024-09-14T06:58:37.800 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T06:58:37.800 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T06:58:37.874 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:58:37.874 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:58:37.874 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:58:37.875 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:58:37.875 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:58:37.875 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:58:37.875 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:58:37.875 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:58:37.875 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:58:37.875 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:58:38.401 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2024-09-14T06:58:38.403 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-14T06:58:38.403 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-14T06:58:38.547 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-14T06:58:38.581 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:38.681 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:38.680+0000 7fdcb67b2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-14T06:58:38.682 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:38.680+0000 7fdcb67b2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-14T06:58:38.682 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:38.680+0000 7fdcb67b2640 -1 bdev(0x55a871631800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-14T06:58:38.682 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:38.680+0000 7fdcb67b2640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-14T06:58:42.093 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-14T06:58:42.123 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:42.215 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:42.214+0000 7f92dd40b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-14T06:58:42.215 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:42.214+0000 7f92dd40b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-14T06:58:42.215 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:42.214+0000 7f92dd40b640 -1 bdev(0x55b825d57800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-14T06:58:42.215 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:42.214+0000 7f92dd40b640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-14T06:58:45.373 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-14T06:58:45.403 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:45.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:45.493+0000 7f1b77b17640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-14T06:58:45.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:45.493+0000 7f1b77b17640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-14T06:58:45.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:45.493+0000 7f1b77b17640 -1 bdev(0x55575c66d800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-14T06:58:45.494 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:45.493+0000 7f1b77b17640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-14T06:58:49.152 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-14T06:58:49.183 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:49.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:49.271+0000 7f63c7aef640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-14T06:58:49.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:49.272+0000 7f63c7aef640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-14T06:58:49.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:49.272+0000 7f63c7aef640 -1 bdev(0x5581a8663800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-14T06:58:49.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-14T06:58:49.272+0000 7f63c7aef640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-14T06:58:52.185 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-14T06:58:52.215 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-14T06:58:52.215 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:52.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-14T06:58:52.247 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:52.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-14T06:58:52.316 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:52.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-14T06:58:52.389 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:52.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-14T06:58:52.457 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:52.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-14T06:58:52.525 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:52.525 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-14T06:58:52.555 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:52.555 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-14T06:58:52.624 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:52.625 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-14T06:58:52.693 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:52.693 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-14T06:58:52.760 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:52.760 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-14T06:58:52.827 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:58:52.827 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-14T06:58:52.847 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:58:52.847 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-14T06:58:52.884 INFO:tasks.ceph:Adding keys to all mons... 2024-09-14T06:58:52.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T06:58:52.889 DEBUG:teuthology.orchestra.run.smithi189:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAMNOVm7sUHNhAArqs9Iw982eIquzKdK4kAPw== 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAONOVmJsv3FRAABwqimJAtMGWIMCbqHiAshA== 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQARNOVmUwDSNRAAh115WdENbA2jGi0Gvl/xCQ== 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAVNOVmXjEQCxAArVVTmftx+BvNV9oDAenjag== 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAYNOVmtAxXDRAAOzNNNn1Xe61cblvenbSmbA== 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-09-14T06:58:52.928 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAMNOVmIwUCOhAAj6WfYZ77DBViz+fVuSOjug== 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAeNOVmQt+bKBAATxnomigUq+ysX3aE3kLuMA== 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAiNOVmsRnQDBAAqk53txV95ruHsTtVX51AHw== 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAlNOVmzvx0HRAABQjGIfjsmEVsEtrfq/mF5g== 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQApNOVmG7VAEBAAJcJFh+nVoes3BGSi7vob5A== 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQANNOVmWU0hARAAMYbh1dQLqsuvYqHBKwWa2A== 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-09-14T06:58:52.929 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQANNOVmAWheBBAAVpma2an7S9+nvgX/1SzUqw== 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr.y] 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAMNOVm7sUHNhAArqs9Iw982eIquzKdK4kAPw== 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.0] 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAONOVmJsv3FRAABwqimJAtMGWIMCbqHiAshA== 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.1] 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQARNOVmUwDSNRAAh115WdENbA2jGi0Gvl/xCQ== 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.2] 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAVNOVmXjEQCxAArVVTmftx+BvNV9oDAenjag== 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.3] 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAYNOVmtAxXDRAAOzNNNn1Xe61cblvenbSmbA== 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr.x] 2024-09-14T06:58:52.949 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAMNOVmIwUCOhAAj6WfYZ77DBViz+fVuSOjug== 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.4] 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAeNOVmQt+bKBAATxnomigUq+ysX3aE3kLuMA== 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.5] 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAiNOVmsRnQDBAAqk53txV95ruHsTtVX51AHw== 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.6] 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAlNOVmzvx0HRAABQjGIfjsmEVsEtrfq/mF5g== 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.7] 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQApNOVmG7VAEBAAJcJFh+nVoes3BGSi7vob5A== 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout:[client.0] 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQANNOVmWU0hARAAMYbh1dQLqsuvYqHBKwWa2A== 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout:[client.1] 2024-09-14T06:58:52.950 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQANNOVmAWheBBAAVpma2an7S9+nvgX/1SzUqw== 2024-09-14T06:58:52.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:58:52.973 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:58:53.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.047 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.103 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.157 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.208 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.210 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:58:53.263 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:58:53.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.317 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.370 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.423 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.476 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:58:53.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:58:53.529 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:58:53.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:58:53.585 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:58:53.639 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-14T06:58:53.640 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-14T06:58:53.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T06:58:53.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-14T06:58:53.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-14T06:58:53.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T06:58:53.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-14T06:58:53.974 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-14T06:58:53.999 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T06:58:54.099 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-14T06:58:54.124 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:54.127 DEBUG:teuthology.orchestra.run.smithi189:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:58:54.180 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-14T06:58:54.180 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-14T06:58:54.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-14T06:58:54.183 INFO:tasks.ceph.mon.a:Started 2024-09-14T06:58:54.183 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-14T06:58:54.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-14T06:58:54.185 INFO:tasks.ceph.mon.c:Started 2024-09-14T06:58:54.185 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-14T06:58:54.185 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-14T06:58:54.225 INFO:tasks.ceph.mon.b:Started 2024-09-14T06:58:54.226 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-14T06:58:54.226 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-14T06:58:54.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-14T06:58:54.228 INFO:tasks.ceph.mgr.y:Started 2024-09-14T06:58:54.228 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-14T06:58:54.229 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-14T06:58:54.231 INFO:tasks.ceph.mgr.x:Started 2024-09-14T06:58:54.231 DEBUG:tasks.ceph:set 0 configs 2024-09-14T06:58:54.231 INFO:tasks.ceph:Setting crush tunables to default 2024-09-14T06:58:54.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-09-14T06:59:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-09-14T06:59:00.455 INFO:tasks.ceph:check_enable_crimson: False 2024-09-14T06:59:00.456 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-14T06:59:00.456 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:59:00.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-14T06:59:00.482 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:59:00.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-14T06:59:00.548 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:59:00.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-14T06:59:00.614 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-14T06:59:00.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-14T06:59:00.680 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:59:00.680 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-14T06:59:00.707 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:59:00.707 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-14T06:59:00.773 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:59:00.773 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-14T06:59:00.839 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-14T06:59:00.840 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-14T06:59:00.904 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new c036038d-cffb-4584-8851-69604a31ee40 0 2024-09-14T06:59:01.124 INFO:teuthology.orchestra.run.smithi189.stdout:0 2024-09-14T06:59:01.136 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 1c80080f-932a-4957-b64b-f30c4bdb9340 1 2024-09-14T06:59:01.321 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-09-14T06:59:01.332 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new f3544d11-8a0f-498c-a03c-5d34134c9773 2 2024-09-14T06:59:01.516 INFO:teuthology.orchestra.run.smithi189.stdout:2 2024-09-14T06:59:01.527 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new d2261455-c91b-414b-925f-388f0e094dbe 3 2024-09-14T06:59:01.712 INFO:teuthology.orchestra.run.smithi189.stdout:3 2024-09-14T06:59:01.724 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 4088d23b-0d06-4415-9eba-6fb64355746c 4 2024-09-14T06:59:01.909 INFO:teuthology.orchestra.run.smithi189.stdout:4 2024-09-14T06:59:01.920 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 5e27cb53-ecce-4061-bc5a-be599bf18cc4 5 2024-09-14T06:59:02.104 INFO:teuthology.orchestra.run.smithi189.stdout:5 2024-09-14T06:59:02.116 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new ceb9a1f1-f54c-48d5-91d8-2287cac96ceb 6 2024-09-14T06:59:02.296 INFO:teuthology.orchestra.run.smithi189.stdout:6 2024-09-14T06:59:02.308 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 85c59cd6-6e2b-4cc0-852d-d1f075a8826d 7 2024-09-14T06:59:02.493 INFO:teuthology.orchestra.run.smithi189.stdout:7 2024-09-14T06:59:02.504 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T06:59:02.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T06:59:02.507 INFO:tasks.ceph.osd.0:Started 2024-09-14T06:59:02.507 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-14T06:59:02.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-14T06:59:02.509 INFO:tasks.ceph.osd.1:Started 2024-09-14T06:59:02.509 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T06:59:02.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T06:59:02.511 INFO:tasks.ceph.osd.2:Started 2024-09-14T06:59:02.511 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-14T06:59:02.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-14T06:59:02.513 INFO:tasks.ceph.osd.3:Started 2024-09-14T06:59:02.514 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T06:59:02.514 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T06:59:02.516 INFO:tasks.ceph.osd.4:Started 2024-09-14T06:59:02.516 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-14T06:59:02.516 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-14T06:59:02.518 INFO:tasks.ceph.osd.5:Started 2024-09-14T06:59:02.518 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-14T06:59:02.518 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-14T06:59:02.521 INFO:tasks.ceph.osd.6:Started 2024-09-14T06:59:02.521 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-14T06:59:02.521 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-14T06:59:02.523 INFO:tasks.ceph.osd.7:Started 2024-09-14T06:59:02.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:59:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:02.726 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6","created":"2024-09-14T06:58:59.479055+0000","modified":"2024-09-14T06:59:02.486396+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T06:59:02.486396+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c036038d-cffb-4584-8851-69604a31ee40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1c80080f-932a-4957-b64b-f30c4bdb9340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f3544d11-8a0f-498c-a03c-5d34134c9773","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d2261455-c91b-414b-925f-388f0e094dbe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4088d23b-0d06-4415-9eba-6fb64355746c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5e27cb53-ecce-4061-bc5a-be599bf18cc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ceb9a1f1-f54c-48d5-91d8-2287cac96ceb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"85c59cd6-6e2b-4cc0-852d-d1f075a8826d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:59:02.737 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-14T06:59:02.737 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-14T06:59:03.542 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:03.541+0000 7fdebbf1e640 -1 Falling back to public interface 2024-09-14T06:59:03.545 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T06:59:03.709 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-14T06:59:03.709 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":10,"fsid":"32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6","created":"2024-09-14T06:58:59.479055+0000","modified":"2024-09-14T06:59:02.486396+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T06:59:02.486396+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c036038d-cffb-4584-8851-69604a31ee40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1c80080f-932a-4957-b64b-f30c4bdb9340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f3544d11-8a0f-498c-a03c-5d34134c9773","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d2261455-c91b-414b-925f-388f0e094dbe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4088d23b-0d06-4415-9eba-6fb64355746c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5e27cb53-ecce-4061-bc5a-be599bf18cc4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ceb9a1f1-f54c-48d5-91d8-2287cac96ceb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"85c59cd6-6e2b-4cc0-852d-d1f075a8826d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:59:03.718 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T06:59:03.984 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:03.983+0000 7f3eadc2e640 -1 Falling back to public interface 2024-09-14T06:59:03.985 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:03.984+0000 7f986de24640 -1 Falling back to public interface 2024-09-14T06:59:03.993 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:03.992+0000 7f7fd49c3640 -1 Falling back to public interface 2024-09-14T06:59:04.001 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:04.000+0000 7f92f4820640 -1 Falling back to public interface 2024-09-14T06:59:04.033 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:04.032+0000 7f400cc19640 -1 Falling back to public interface 2024-09-14T06:59:04.530 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:04.529+0000 7f40850e4640 -1 Falling back to public interface 2024-09-14T06:59:04.539 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:04.538+0000 7f20a3a17640 -1 Falling back to public interface 2024-09-14T06:59:04.756 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-14T06:59:04.755+0000 7f9f1297d640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-14T06:59:04.756 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-14T06:59:04.756+0000 7f9f1297d640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-14T06:59:07.314 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:07.313+0000 7fdebbf1e640 -1 osd.7 0 log_to_monitors true 2024-09-14T06:59:07.734 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:07.733+0000 7f3eadc2e640 -1 osd.3 0 log_to_monitors true 2024-09-14T06:59:07.801 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:07.800+0000 7f986de24640 -1 osd.1 0 log_to_monitors true 2024-09-14T06:59:07.814 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:07.813+0000 7f400cc19640 -1 osd.4 0 log_to_monitors true 2024-09-14T06:59:08.093 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:08.092+0000 7f7fd49c3640 -1 osd.0 0 log_to_monitors true 2024-09-14T06:59:08.095 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:08.095+0000 7f92f4820640 -1 osd.2 0 log_to_monitors true 2024-09-14T06:59:08.290 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:08.289+0000 7f40850e4640 -1 osd.5 0 log_to_monitors true 2024-09-14T06:59:09.081 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:09.080+0000 7f20a3a17640 -1 osd.6 0 log_to_monitors true 2024-09-14T06:59:09.222 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:09.222+0000 7fdeb86c1640 -1 osd.7 0 waiting for initial osdmap 2024-09-14T06:59:09.237 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:09.236+0000 7fdeafccc640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:59:10.381 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:10.380+0000 7f4008ba8640 -1 osd.4 0 waiting for initial osdmap 2024-09-14T06:59:10.397 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:10.397+0000 7f40009c7640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:59:10.421 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:10.420+0000 7f4081887640 -1 osd.5 0 waiting for initial osdmap 2024-09-14T06:59:10.438 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:10.437+0000 7f4078e92640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:59:10.526 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T06:59:10.702 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:10.700+0000 7f9869db3640 -1 osd.1 0 waiting for initial osdmap 2024-09-14T06:59:10.703 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:10.702+0000 7f7fd1166640 -1 osd.0 0 waiting for initial osdmap 2024-09-14T06:59:10.722 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:10.721+0000 7f9861bd2640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:59:10.723 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:10.722+0000 7f7fc8771640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:59:10.729 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:10.729+0000 7f3ea9185640 -1 osd.3 0 waiting for initial osdmap 2024-09-14T06:59:10.730 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:10.729+0000 7f92f07af640 -1 osd.2 0 waiting for initial osdmap 2024-09-14T06:59:10.741 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:10.740+0000 7f92e85ce640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:59:10.741 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:10.740+0000 7f3ea0fa4640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:59:10.909 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-14T06:59:10.909 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":14,"fsid":"32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6","created":"2024-09-14T06:58:59.479055+0000","modified":"2024-09-14T06:59:10.511028+0000","last_up_change":"2024-09-14T06:59:10.511028+0000","last_in_change":"2024-09-14T06:59:02.486396+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c036038d-cffb-4584-8851-69604a31ee40","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1c80080f-932a-4957-b64b-f30c4bdb9340","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f3544d11-8a0f-498c-a03c-5d34134c9773","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d2261455-c91b-414b-925f-388f0e094dbe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4088d23b-0d06-4415-9eba-6fb64355746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6811","nonce":2283562461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6813","nonce":2283562461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6817","nonce":2283562461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6815","nonce":2283562461}]},"public_addr":"172.21.15.189:6811/2283562461","cluster_addr":"172.21.15.189:6813/2283562461","heartbeat_back_addr":"172.21.15.189:6817/2283562461","heartbeat_front_addr":"172.21.15.189:6815/2283562461","state":["exists","up"]},{"osd":5,"uuid":"5e27cb53-ecce-4061-bc5a-be599bf18cc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6819","nonce":2212769626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6821","nonce":2212769626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6825","nonce":2212769626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6823","nonce":2212769626}]},"public_addr":"172.21.15.189:6819/2212769626","cluster_addr":"172.21.15.189:6821/2212769626","heartbeat_back_addr":"172.21.15.189:6825/2212769626","heartbeat_front_addr":"172.21.15.189:6823/2212769626","state":["exists","up"]},{"osd":6,"uuid":"ceb9a1f1-f54c-48d5-91d8-2287cac96ceb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"85c59cd6-6e2b-4cc0-852d-d1f075a8826d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6801","nonce":2162534312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6803","nonce":2162534312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6807","nonce":2162534312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6805","nonce":2162534312}]},"public_addr":"172.21.15.189:6801/2162534312","cluster_addr":"172.21.15.189:6803/2162534312","heartbeat_back_addr":"172.21.15.189:6807/2162534312","heartbeat_front_addr":"172.21.15.189:6805/2162534312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.300043+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:59:10.921 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-09-14T06:59:11.230 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:11.229+0000 7f209f401640 -1 osd.6 0 waiting for initial osdmap 2024-09-14T06:59:11.248 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:11.247+0000 7f2097220640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:59:17.728 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T06:59:18.030 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-14T06:59:18.030 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":18,"fsid":"32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6","created":"2024-09-14T06:58:59.479055+0000","modified":"2024-09-14T06:59:14.729991+0000","last_up_change":"2024-09-14T06:59:11.718934+0000","last_in_change":"2024-09-14T06:59:02.486396+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:59:11.765419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c036038d-cffb-4584-8851-69604a31ee40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6817","nonce":2081065366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6819","nonce":2081065366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6823","nonce":2081065366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6821","nonce":2081065366}]},"public_addr":"172.21.15.7:6817/2081065366","cluster_addr":"172.21.15.7:6819/2081065366","heartbeat_back_addr":"172.21.15.7:6823/2081065366","heartbeat_front_addr":"172.21.15.7:6821/2081065366","state":["exists","up"]},{"osd":1,"uuid":"1c80080f-932a-4957-b64b-f30c4bdb9340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6806","nonce":1688050909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6810","nonce":1688050909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6815","nonce":1688050909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6813","nonce":1688050909}]},"public_addr":"172.21.15.7:6806/1688050909","cluster_addr":"172.21.15.7:6810/1688050909","heartbeat_back_addr":"172.21.15.7:6815/1688050909","heartbeat_front_addr":"172.21.15.7:6813/1688050909","state":["exists","up"]},{"osd":2,"uuid":"f3544d11-8a0f-498c-a03c-5d34134c9773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6825","nonce":1872259061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6827","nonce":1872259061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6831","nonce":1872259061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6829","nonce":1872259061}]},"public_addr":"172.21.15.7:6825/1872259061","cluster_addr":"172.21.15.7:6827/1872259061","heartbeat_back_addr":"172.21.15.7:6831/1872259061","heartbeat_front_addr":"172.21.15.7:6829/1872259061","state":["exists","up"]},{"osd":3,"uuid":"d2261455-c91b-414b-925f-388f0e094dbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6801","nonce":1193301030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6803","nonce":1193301030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6811","nonce":1193301030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6807","nonce":1193301030}]},"public_addr":"172.21.15.7:6801/1193301030","cluster_addr":"172.21.15.7:6803/1193301030","heartbeat_back_addr":"172.21.15.7:6811/1193301030","heartbeat_front_addr":"172.21.15.7:6807/1193301030","state":["exists","up"]},{"osd":4,"uuid":"4088d23b-0d06-4415-9eba-6fb64355746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6811","nonce":2283562461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6813","nonce":2283562461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6817","nonce":2283562461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6815","nonce":2283562461}]},"public_addr":"172.21.15.189:6811/2283562461","cluster_addr":"172.21.15.189:6813/2283562461","heartbeat_back_addr":"172.21.15.189:6817/2283562461","heartbeat_front_addr":"172.21.15.189:6815/2283562461","state":["exists","up"]},{"osd":5,"uuid":"5e27cb53-ecce-4061-bc5a-be599bf18cc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6819","nonce":2212769626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6821","nonce":2212769626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6825","nonce":2212769626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6823","nonce":2212769626}]},"public_addr":"172.21.15.189:6819/2212769626","cluster_addr":"172.21.15.189:6821/2212769626","heartbeat_back_addr":"172.21.15.189:6825/2212769626","heartbeat_front_addr":"172.21.15.189:6823/2212769626","state":["exists","up"]},{"osd":6,"uuid":"ceb9a1f1-f54c-48d5-91d8-2287cac96ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6827","nonce":3197231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6829","nonce":3197231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6833","nonce":3197231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6831","nonce":3197231389}]},"public_addr":"172.21.15.189:6827/3197231389","cluster_addr":"172.21.15.189:6829/3197231389","heartbeat_back_addr":"172.21.15.189:6833/3197231389","heartbeat_front_addr":"172.21.15.189:6831/3197231389","state":["exists","up"]},{"osd":7,"uuid":"85c59cd6-6e2b-4cc0-852d-d1f075a8826d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6801","nonce":2162534312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6803","nonce":2162534312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6807","nonce":2162534312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6805","nonce":2162534312}]},"public_addr":"172.21.15.189:6801/2162534312","cluster_addr":"172.21.15.189:6803/2162534312","heartbeat_back_addr":"172.21.15.189:6807/2162534312","heartbeat_front_addr":"172.21.15.189:6805/2162534312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.064941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.780598+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.127700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.730995+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.864423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.329361+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:10.107581+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.300043+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:59:18.041 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-14T06:59:18.041 INFO:tasks.ceph:Creating RBD pool 2024-09-14T06:59:18.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-14T06:59:18.748 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'rbd' created 2024-09-14T06:59:18.763 DEBUG:teuthology.orchestra.run.smithi007:> rbd --cluster ceph pool init rbd 2024-09-14T06:59:21.774 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-14T06:59:21.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-14T06:59:21.776 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-14T06:59:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:22.100 INFO:teuthology.orchestra.run.smithi007.stdout:[{"version":9,"timestamp":"2024-09-14T06:59:11.231582+0000","name":"","changes":[{"name":"osd.6/osd_mclock_max_capacity_iops_ssd","new_value":"7516.510711"}]}] 2024-09-14T06:59:22.100 INFO:tasks.ceph_manager:config epoch is 9 2024-09-14T06:59:22.100 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-14T06:59:22.100 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-14T06:59:22.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-14T06:59:22.454 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:22.465 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":2188983240},{"type":"v1","addr":"172.21.15.189:6809","nonce":2188983240}]},"active_addr":"172.21.15.189:6809/2188983240","active_change":"2024-09-14T06:59:03.742807+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":3611044437}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":2961180613}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":1177705200}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":3539570407}]}]} 2024-09-14T06:59:22.469 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-14T06:59:22.469 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T06:59:22.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:59:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:22.769 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6","created":"2024-09-14T06:58:59.479055+0000","modified":"2024-09-14T06:59:21.760131+0000","last_up_change":"2024-09-14T06:59:11.718934+0000","last_in_change":"2024-09-14T06:59:02.486396+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:59:11.765419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:59:18.343475+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c036038d-cffb-4584-8851-69604a31ee40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6817","nonce":2081065366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6819","nonce":2081065366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6823","nonce":2081065366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6821","nonce":2081065366}]},"public_addr":"172.21.15.7:6817/2081065366","cluster_addr":"172.21.15.7:6819/2081065366","heartbeat_back_addr":"172.21.15.7:6823/2081065366","heartbeat_front_addr":"172.21.15.7:6821/2081065366","state":["exists","up"]},{"osd":1,"uuid":"1c80080f-932a-4957-b64b-f30c4bdb9340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6806","nonce":1688050909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6810","nonce":1688050909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6815","nonce":1688050909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6813","nonce":1688050909}]},"public_addr":"172.21.15.7:6806/1688050909","cluster_addr":"172.21.15.7:6810/1688050909","heartbeat_back_addr":"172.21.15.7:6815/1688050909","heartbeat_front_addr":"172.21.15.7:6813/1688050909","state":["exists","up"]},{"osd":2,"uuid":"f3544d11-8a0f-498c-a03c-5d34134c9773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6825","nonce":1872259061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6827","nonce":1872259061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6831","nonce":1872259061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6829","nonce":1872259061}]},"public_addr":"172.21.15.7:6825/1872259061","cluster_addr":"172.21.15.7:6827/1872259061","heartbeat_back_addr":"172.21.15.7:6831/1872259061","heartbeat_front_addr":"172.21.15.7:6829/1872259061","state":["exists","up"]},{"osd":3,"uuid":"d2261455-c91b-414b-925f-388f0e094dbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6801","nonce":1193301030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6803","nonce":1193301030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6811","nonce":1193301030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6807","nonce":1193301030}]},"public_addr":"172.21.15.7:6801/1193301030","cluster_addr":"172.21.15.7:6803/1193301030","heartbeat_back_addr":"172.21.15.7:6811/1193301030","heartbeat_front_addr":"172.21.15.7:6807/1193301030","state":["exists","up"]},{"osd":4,"uuid":"4088d23b-0d06-4415-9eba-6fb64355746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6811","nonce":2283562461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6813","nonce":2283562461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6817","nonce":2283562461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6815","nonce":2283562461}]},"public_addr":"172.21.15.189:6811/2283562461","cluster_addr":"172.21.15.189:6813/2283562461","heartbeat_back_addr":"172.21.15.189:6817/2283562461","heartbeat_front_addr":"172.21.15.189:6815/2283562461","state":["exists","up"]},{"osd":5,"uuid":"5e27cb53-ecce-4061-bc5a-be599bf18cc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6819","nonce":2212769626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6821","nonce":2212769626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6825","nonce":2212769626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6823","nonce":2212769626}]},"public_addr":"172.21.15.189:6819/2212769626","cluster_addr":"172.21.15.189:6821/2212769626","heartbeat_back_addr":"172.21.15.189:6825/2212769626","heartbeat_front_addr":"172.21.15.189:6823/2212769626","state":["exists","up"]},{"osd":6,"uuid":"ceb9a1f1-f54c-48d5-91d8-2287cac96ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6827","nonce":3197231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6829","nonce":3197231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6833","nonce":3197231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6831","nonce":3197231389}]},"public_addr":"172.21.15.189:6827/3197231389","cluster_addr":"172.21.15.189:6829/3197231389","heartbeat_back_addr":"172.21.15.189:6833/3197231389","heartbeat_front_addr":"172.21.15.189:6831/3197231389","state":["exists","up"]},{"osd":7,"uuid":"85c59cd6-6e2b-4cc0-852d-d1f075a8826d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6801","nonce":2162534312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6803","nonce":2162534312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6807","nonce":2162534312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6805","nonce":2162534312}]},"public_addr":"172.21.15.189:6801/2162534312","cluster_addr":"172.21.15.189:6803/2162534312","heartbeat_back_addr":"172.21.15.189:6807/2162534312","heartbeat_front_addr":"172.21.15.189:6805/2162534312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.064941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.780598+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.127700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.730995+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.864423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.329361+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:10.107581+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.300043+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:59:22.781 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T06:59:22.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:59:23.081 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:23.081 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6","created":"2024-09-14T06:58:59.479055+0000","modified":"2024-09-14T06:59:21.760131+0000","last_up_change":"2024-09-14T06:59:11.718934+0000","last_in_change":"2024-09-14T06:59:02.486396+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:59:11.765419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:59:18.343475+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c036038d-cffb-4584-8851-69604a31ee40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6817","nonce":2081065366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6819","nonce":2081065366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6823","nonce":2081065366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6821","nonce":2081065366}]},"public_addr":"172.21.15.7:6817/2081065366","cluster_addr":"172.21.15.7:6819/2081065366","heartbeat_back_addr":"172.21.15.7:6823/2081065366","heartbeat_front_addr":"172.21.15.7:6821/2081065366","state":["exists","up"]},{"osd":1,"uuid":"1c80080f-932a-4957-b64b-f30c4bdb9340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6806","nonce":1688050909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6810","nonce":1688050909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6815","nonce":1688050909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6813","nonce":1688050909}]},"public_addr":"172.21.15.7:6806/1688050909","cluster_addr":"172.21.15.7:6810/1688050909","heartbeat_back_addr":"172.21.15.7:6815/1688050909","heartbeat_front_addr":"172.21.15.7:6813/1688050909","state":["exists","up"]},{"osd":2,"uuid":"f3544d11-8a0f-498c-a03c-5d34134c9773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6825","nonce":1872259061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6827","nonce":1872259061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6831","nonce":1872259061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6829","nonce":1872259061}]},"public_addr":"172.21.15.7:6825/1872259061","cluster_addr":"172.21.15.7:6827/1872259061","heartbeat_back_addr":"172.21.15.7:6831/1872259061","heartbeat_front_addr":"172.21.15.7:6829/1872259061","state":["exists","up"]},{"osd":3,"uuid":"d2261455-c91b-414b-925f-388f0e094dbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6801","nonce":1193301030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6803","nonce":1193301030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6811","nonce":1193301030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6807","nonce":1193301030}]},"public_addr":"172.21.15.7:6801/1193301030","cluster_addr":"172.21.15.7:6803/1193301030","heartbeat_back_addr":"172.21.15.7:6811/1193301030","heartbeat_front_addr":"172.21.15.7:6807/1193301030","state":["exists","up"]},{"osd":4,"uuid":"4088d23b-0d06-4415-9eba-6fb64355746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6811","nonce":2283562461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6813","nonce":2283562461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6817","nonce":2283562461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6815","nonce":2283562461}]},"public_addr":"172.21.15.189:6811/2283562461","cluster_addr":"172.21.15.189:6813/2283562461","heartbeat_back_addr":"172.21.15.189:6817/2283562461","heartbeat_front_addr":"172.21.15.189:6815/2283562461","state":["exists","up"]},{"osd":5,"uuid":"5e27cb53-ecce-4061-bc5a-be599bf18cc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6819","nonce":2212769626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6821","nonce":2212769626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6825","nonce":2212769626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6823","nonce":2212769626}]},"public_addr":"172.21.15.189:6819/2212769626","cluster_addr":"172.21.15.189:6821/2212769626","heartbeat_back_addr":"172.21.15.189:6825/2212769626","heartbeat_front_addr":"172.21.15.189:6823/2212769626","state":["exists","up"]},{"osd":6,"uuid":"ceb9a1f1-f54c-48d5-91d8-2287cac96ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6827","nonce":3197231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6829","nonce":3197231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6833","nonce":3197231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6831","nonce":3197231389}]},"public_addr":"172.21.15.189:6827/3197231389","cluster_addr":"172.21.15.189:6829/3197231389","heartbeat_back_addr":"172.21.15.189:6833/3197231389","heartbeat_front_addr":"172.21.15.189:6831/3197231389","state":["exists","up"]},{"osd":7,"uuid":"85c59cd6-6e2b-4cc0-852d-d1f075a8826d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6801","nonce":2162534312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6803","nonce":2162534312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6807","nonce":2162534312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6805","nonce":2162534312}]},"public_addr":"172.21.15.189:6801/2162534312","cluster_addr":"172.21.15.189:6803/2162534312","heartbeat_back_addr":"172.21.15.189:6807/2162534312","heartbeat_front_addr":"172.21.15.189:6805/2162534312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.064941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.780598+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.127700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.730995+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.864423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.329361+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:10.107581+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.300043+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:59:23.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T06:59:23.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T06:59:23.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T06:59:23.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T06:59:23.096 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T06:59:23.096 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T06:59:23.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T06:59:23.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T06:59:23.330 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-14T06:59:23.330 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T06:59:23.351 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-14T06:59:23.352 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-14T06:59:23.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T06:59:23.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T06:59:23.353 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-14T06:59:23.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T06:59:23.354 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-14T06:59:23.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T06:59:23.355 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-14T06:59:23.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T06:59:23.357 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-14T06:59:23.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:59:23.358 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-09-14T06:59:23.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T06:59:23.904 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-14T06:59:23.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.2 2024-09-14T06:59:23.926 DEBUG:teuthology.parallel:result is None 2024-09-14T06:59:23.964 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-14T06:59:23.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.6 2024-09-14T06:59:23.978 DEBUG:teuthology.parallel:result is None 2024-09-14T06:59:24.019 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-14T06:59:24.022 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-14T06:59:24.024 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-14T06:59:24.031 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-14T06:59:24.033 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-09-14T06:59:24.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2024-09-14T06:59:24.036 DEBUG:teuthology.parallel:result is None 2024-09-14T06:59:24.038 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-14T06:59:24.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.0 2024-09-14T06:59:24.040 DEBUG:teuthology.parallel:result is None 2024-09-14T06:59:24.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.1 2024-09-14T06:59:24.043 DEBUG:teuthology.parallel:result is None 2024-09-14T06:59:24.052 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-09-14T06:59:24.052 DEBUG:teuthology.parallel:result is None 2024-09-14T06:59:24.054 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.5 2024-09-14T06:59:24.054 DEBUG:teuthology.parallel:result is None 2024-09-14T06:59:24.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-09-14T06:59:24.055 DEBUG:teuthology.parallel:result is None 2024-09-14T06:59:24.055 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T06:59:24.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:59:24.393 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:24.393 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T06:59:24.405 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-14T06:59:24.273440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":1892352,"data_stored":1788374,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.523547"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.767340+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T06:59:21.767340+0000","last_peered":"2024-09-14T06:59:21.767340+0000","last_clean":"2024-09-14T06:59:21.767340+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T06:59:21.767340+0000","last_undegraded":"2024-09-14T06:59:21.767340+0000","last_fullsized":"2024-09-14T06:59:21.767340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.902022+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T06:59:21.902022+0000","last_peered":"2024-09-14T06:59:21.902022+0000","last_clean":"2024-09-14T06:59:21.902022+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T06:59:21.902022+0000","last_undegraded":"2024-09-14T06:59:21.902022+0000","last_fullsized":"2024-09-14T06:59:21.902022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.766844+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T06:59:21.766844+0000","last_peered":"2024-09-14T06:59:21.766844+0000","last_clean":"2024-09-14T06:59:21.766844+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T06:59:21.766844+0000","last_undegraded":"2024-09-14T06:59:21.766844+0000","last_fullsized":"2024-09-14T06:59:21.766844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.901815+0000","last_change":"2024-09-14T06:59:21.902140+0000","last_active":"2024-09-14T06:59:21.901815+0000","last_peered":"2024-09-14T06:59:21.901815+0000","last_clean":"2024-09-14T06:59:21.901815+0000","last_became_active":"2024-09-14T06:59:20.121600+0000","last_became_peered":"2024-09-14T06:59:20.121600+0000","last_unstale":"2024-09-14T06:59:21.901815+0000","last_undegraded":"2024-09-14T06:59:21.901815+0000","last_fullsized":"2024-09-14T06:59:21.901815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:29:52.276761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.769464+0000","last_change":"2024-09-14T06:59:21.767686+0000","last_active":"2024-09-14T06:59:21.769464+0000","last_peered":"2024-09-14T06:59:21.769464+0000","last_clean":"2024-09-14T06:59:21.769464+0000","last_became_active":"2024-09-14T06:59:20.121991+0000","last_became_peered":"2024-09-14T06:59:20.121991+0000","last_unstale":"2024-09-14T06:59:21.769464+0000","last_undegraded":"2024-09-14T06:59:21.769464+0000","last_fullsized":"2024-09-14T06:59:21.769464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:44.196704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.902227+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T06:59:21.902227+0000","last_peered":"2024-09-14T06:59:21.902227+0000","last_clean":"2024-09-14T06:59:21.902227+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T06:59:21.902227+0000","last_undegraded":"2024-09-14T06:59:21.902227+0000","last_fullsized":"2024-09-14T06:59:21.902227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.766791+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T06:59:21.766791+0000","last_peered":"2024-09-14T06:59:21.766791+0000","last_clean":"2024-09-14T06:59:21.766791+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T06:59:21.766791+0000","last_undegraded":"2024-09-14T06:59:21.766791+0000","last_fullsized":"2024-09-14T06:59:21.766791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.767446+0000","last_change":"2024-09-14T06:59:21.767446+0000","last_active":"2024-09-14T06:59:21.767446+0000","last_peered":"2024-09-14T06:59:21.767446+0000","last_clean":"2024-09-14T06:59:21.767446+0000","last_became_active":"2024-09-14T06:59:20.120193+0000","last_became_peered":"2024-09-14T06:59:20.120193+0000","last_unstale":"2024-09-14T06:59:21.767446+0000","last_undegraded":"2024-09-14T06:59:21.767446+0000","last_fullsized":"2024-09-14T06:59:21.767446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:18.122142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.901048+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T06:59:21.901048+0000","last_peered":"2024-09-14T06:59:21.901048+0000","last_clean":"2024-09-14T06:59:21.901048+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T06:59:21.901048+0000","last_undegraded":"2024-09-14T06:59:21.901048+0000","last_fullsized":"2024-09-14T06:59:21.901048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":666318,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":666318,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:59:24.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:59:24.700 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:24.700 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T06:59:24.712 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-14T06:59:24.273440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":1892352,"data_stored":1788374,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.523547"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.767340+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T06:59:21.767340+0000","last_peered":"2024-09-14T06:59:21.767340+0000","last_clean":"2024-09-14T06:59:21.767340+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T06:59:21.767340+0000","last_undegraded":"2024-09-14T06:59:21.767340+0000","last_fullsized":"2024-09-14T06:59:21.767340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.902022+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T06:59:21.902022+0000","last_peered":"2024-09-14T06:59:21.902022+0000","last_clean":"2024-09-14T06:59:21.902022+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T06:59:21.902022+0000","last_undegraded":"2024-09-14T06:59:21.902022+0000","last_fullsized":"2024-09-14T06:59:21.902022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.766844+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T06:59:21.766844+0000","last_peered":"2024-09-14T06:59:21.766844+0000","last_clean":"2024-09-14T06:59:21.766844+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T06:59:21.766844+0000","last_undegraded":"2024-09-14T06:59:21.766844+0000","last_fullsized":"2024-09-14T06:59:21.766844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.901815+0000","last_change":"2024-09-14T06:59:21.902140+0000","last_active":"2024-09-14T06:59:21.901815+0000","last_peered":"2024-09-14T06:59:21.901815+0000","last_clean":"2024-09-14T06:59:21.901815+0000","last_became_active":"2024-09-14T06:59:20.121600+0000","last_became_peered":"2024-09-14T06:59:20.121600+0000","last_unstale":"2024-09-14T06:59:21.901815+0000","last_undegraded":"2024-09-14T06:59:21.901815+0000","last_fullsized":"2024-09-14T06:59:21.901815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:29:52.276761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.769464+0000","last_change":"2024-09-14T06:59:21.767686+0000","last_active":"2024-09-14T06:59:21.769464+0000","last_peered":"2024-09-14T06:59:21.769464+0000","last_clean":"2024-09-14T06:59:21.769464+0000","last_became_active":"2024-09-14T06:59:20.121991+0000","last_became_peered":"2024-09-14T06:59:20.121991+0000","last_unstale":"2024-09-14T06:59:21.769464+0000","last_undegraded":"2024-09-14T06:59:21.769464+0000","last_fullsized":"2024-09-14T06:59:21.769464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:44.196704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.902227+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T06:59:21.902227+0000","last_peered":"2024-09-14T06:59:21.902227+0000","last_clean":"2024-09-14T06:59:21.902227+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T06:59:21.902227+0000","last_undegraded":"2024-09-14T06:59:21.902227+0000","last_fullsized":"2024-09-14T06:59:21.902227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.766791+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T06:59:21.766791+0000","last_peered":"2024-09-14T06:59:21.766791+0000","last_clean":"2024-09-14T06:59:21.766791+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T06:59:21.766791+0000","last_undegraded":"2024-09-14T06:59:21.766791+0000","last_fullsized":"2024-09-14T06:59:21.766791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.767446+0000","last_change":"2024-09-14T06:59:21.767446+0000","last_active":"2024-09-14T06:59:21.767446+0000","last_peered":"2024-09-14T06:59:21.767446+0000","last_clean":"2024-09-14T06:59:21.767446+0000","last_became_active":"2024-09-14T06:59:20.120193+0000","last_became_peered":"2024-09-14T06:59:20.120193+0000","last_unstale":"2024-09-14T06:59:21.767446+0000","last_undegraded":"2024-09-14T06:59:21.767446+0000","last_fullsized":"2024-09-14T06:59:21.767446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:18.122142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.901048+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T06:59:21.901048+0000","last_peered":"2024-09-14T06:59:21.901048+0000","last_clean":"2024-09-14T06:59:21.901048+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T06:59:21.901048+0000","last_undegraded":"2024-09-14T06:59:21.901048+0000","last_fullsized":"2024-09-14T06:59:21.901048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":666318,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":666318,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:59:24.713 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T06:59:24.713 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-14T06:59:24.714 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-14T06:59:24.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-14T06:59:25.054 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:25.054 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-14T06:59:25.067 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-14T06:59:25.068 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-14T06:59:25.474 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-14T06:59:25.475 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}} 2024-09-14T06:59:25.475 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-14T06:59:25.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-14T06:59:25.798 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-14T06:59:25.798 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T06:59:25.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:59:26.093 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:26.094 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T06:59:26.105 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-14T06:59:25.751247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":1892352,"data_stored":1788374,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.999242"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.767340+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T06:59:21.767340+0000","last_peered":"2024-09-14T06:59:21.767340+0000","last_clean":"2024-09-14T06:59:21.767340+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T06:59:21.767340+0000","last_undegraded":"2024-09-14T06:59:21.767340+0000","last_fullsized":"2024-09-14T06:59:21.767340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.902022+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T06:59:21.902022+0000","last_peered":"2024-09-14T06:59:21.902022+0000","last_clean":"2024-09-14T06:59:21.902022+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T06:59:21.902022+0000","last_undegraded":"2024-09-14T06:59:21.902022+0000","last_fullsized":"2024-09-14T06:59:21.902022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.766844+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T06:59:21.766844+0000","last_peered":"2024-09-14T06:59:21.766844+0000","last_clean":"2024-09-14T06:59:21.766844+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T06:59:21.766844+0000","last_undegraded":"2024-09-14T06:59:21.766844+0000","last_fullsized":"2024-09-14T06:59:21.766844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T06:59:25.294244+0000","last_change":"2024-09-14T06:59:21.902140+0000","last_active":"2024-09-14T06:59:25.294244+0000","last_peered":"2024-09-14T06:59:25.294244+0000","last_clean":"2024-09-14T06:59:25.294244+0000","last_became_active":"2024-09-14T06:59:20.121600+0000","last_became_peered":"2024-09-14T06:59:20.121600+0000","last_unstale":"2024-09-14T06:59:25.294244+0000","last_undegraded":"2024-09-14T06:59:25.294244+0000","last_fullsized":"2024-09-14T06:59:25.294244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:29:52.276761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.769464+0000","last_change":"2024-09-14T06:59:21.767686+0000","last_active":"2024-09-14T06:59:21.769464+0000","last_peered":"2024-09-14T06:59:21.769464+0000","last_clean":"2024-09-14T06:59:21.769464+0000","last_became_active":"2024-09-14T06:59:20.121991+0000","last_became_peered":"2024-09-14T06:59:20.121991+0000","last_unstale":"2024-09-14T06:59:21.769464+0000","last_undegraded":"2024-09-14T06:59:21.769464+0000","last_fullsized":"2024-09-14T06:59:21.769464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:44.196704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.902227+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T06:59:21.902227+0000","last_peered":"2024-09-14T06:59:21.902227+0000","last_clean":"2024-09-14T06:59:21.902227+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T06:59:21.902227+0000","last_undegraded":"2024-09-14T06:59:21.902227+0000","last_fullsized":"2024-09-14T06:59:21.902227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.766791+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T06:59:21.766791+0000","last_peered":"2024-09-14T06:59:21.766791+0000","last_clean":"2024-09-14T06:59:21.766791+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T06:59:21.766791+0000","last_undegraded":"2024-09-14T06:59:21.766791+0000","last_fullsized":"2024-09-14T06:59:21.766791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.767446+0000","last_change":"2024-09-14T06:59:21.767446+0000","last_active":"2024-09-14T06:59:21.767446+0000","last_peered":"2024-09-14T06:59:21.767446+0000","last_clean":"2024-09-14T06:59:21.767446+0000","last_became_active":"2024-09-14T06:59:20.120193+0000","last_became_peered":"2024-09-14T06:59:20.120193+0000","last_unstale":"2024-09-14T06:59:21.767446+0000","last_undegraded":"2024-09-14T06:59:21.767446+0000","last_fullsized":"2024-09-14T06:59:21.767446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:18.122142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.901048+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T06:59:21.901048+0000","last_peered":"2024-09-14T06:59:21.901048+0000","last_clean":"2024-09-14T06:59:21.901048+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T06:59:21.901048+0000","last_undegraded":"2024-09-14T06:59:21.901048+0000","last_fullsized":"2024-09-14T06:59:21.901048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":666318,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":666318,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:59:26.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:59:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T06:59:26.412 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-14T06:59:25.751247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":1848,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":1892352,"data_stored":1788374,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.999242"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.767340+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T06:59:21.767340+0000","last_peered":"2024-09-14T06:59:21.767340+0000","last_clean":"2024-09-14T06:59:21.767340+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T06:59:21.767340+0000","last_undegraded":"2024-09-14T06:59:21.767340+0000","last_fullsized":"2024-09-14T06:59:21.767340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.902022+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T06:59:21.902022+0000","last_peered":"2024-09-14T06:59:21.902022+0000","last_clean":"2024-09-14T06:59:21.902022+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T06:59:21.902022+0000","last_undegraded":"2024-09-14T06:59:21.902022+0000","last_fullsized":"2024-09-14T06:59:21.902022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.766844+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T06:59:21.766844+0000","last_peered":"2024-09-14T06:59:21.766844+0000","last_clean":"2024-09-14T06:59:21.766844+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T06:59:21.766844+0000","last_undegraded":"2024-09-14T06:59:21.766844+0000","last_fullsized":"2024-09-14T06:59:21.766844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T06:59:25.294244+0000","last_change":"2024-09-14T06:59:21.902140+0000","last_active":"2024-09-14T06:59:25.294244+0000","last_peered":"2024-09-14T06:59:25.294244+0000","last_clean":"2024-09-14T06:59:25.294244+0000","last_became_active":"2024-09-14T06:59:20.121600+0000","last_became_peered":"2024-09-14T06:59:20.121600+0000","last_unstale":"2024-09-14T06:59:25.294244+0000","last_undegraded":"2024-09-14T06:59:25.294244+0000","last_fullsized":"2024-09-14T06:59:25.294244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:29:52.276761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.769464+0000","last_change":"2024-09-14T06:59:21.767686+0000","last_active":"2024-09-14T06:59:21.769464+0000","last_peered":"2024-09-14T06:59:21.769464+0000","last_clean":"2024-09-14T06:59:21.769464+0000","last_became_active":"2024-09-14T06:59:20.121991+0000","last_became_peered":"2024-09-14T06:59:20.121991+0000","last_unstale":"2024-09-14T06:59:21.769464+0000","last_undegraded":"2024-09-14T06:59:21.769464+0000","last_fullsized":"2024-09-14T06:59:21.769464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:44.196704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.902227+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T06:59:21.902227+0000","last_peered":"2024-09-14T06:59:21.902227+0000","last_clean":"2024-09-14T06:59:21.902227+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T06:59:21.902227+0000","last_undegraded":"2024-09-14T06:59:21.902227+0000","last_fullsized":"2024-09-14T06:59:21.902227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.766791+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T06:59:21.766791+0000","last_peered":"2024-09-14T06:59:21.766791+0000","last_clean":"2024-09-14T06:59:21.766791+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T06:59:21.766791+0000","last_undegraded":"2024-09-14T06:59:21.766791+0000","last_fullsized":"2024-09-14T06:59:21.766791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.767446+0000","last_change":"2024-09-14T06:59:21.767446+0000","last_active":"2024-09-14T06:59:21.767446+0000","last_peered":"2024-09-14T06:59:21.767446+0000","last_clean":"2024-09-14T06:59:21.767446+0000","last_became_active":"2024-09-14T06:59:20.120193+0000","last_became_peered":"2024-09-14T06:59:20.120193+0000","last_unstale":"2024-09-14T06:59:21.767446+0000","last_undegraded":"2024-09-14T06:59:21.767446+0000","last_fullsized":"2024-09-14T06:59:21.767446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:18.122142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"18'72","reported_seq":110,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:59:21.901048+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T06:59:21.901048+0000","last_peered":"2024-09-14T06:59:21.901048+0000","last_clean":"2024-09-14T06:59:21.901048+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T06:59:21.901048+0000","last_undegraded":"2024-09-14T06:59:21.901048+0000","last_fullsized":"2024-09-14T06:59:21.901048+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31460,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":278528,"data_stored":666318,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":666318,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:59:26.413 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T06:59:26.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-14T06:59:26.712 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 23 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-14T06:58:59.479055+0000 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-14T06:59:24.270379+0000 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 6 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client jewel 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-09-14T06:59:26.713 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 4.98 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 8 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6816/2081065366,v1:172.21.15.7:6817/2081065366] [v2:172.21.15.7:6818/2081065366,v1:172.21.15.7:6819/2081065366] exists,up c036038d-cffb-4584-8851-69604a31ee40 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6804/1688050909,v1:172.21.15.7:6806/1688050909] [v2:172.21.15.7:6808/1688050909,v1:172.21.15.7:6810/1688050909] exists,up 1c80080f-932a-4957-b64b-f30c4bdb9340 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6824/1872259061,v1:172.21.15.7:6825/1872259061] [v2:172.21.15.7:6826/1872259061,v1:172.21.15.7:6827/1872259061] exists,up f3544d11-8a0f-498c-a03c-5d34134c9773 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6800/1193301030,v1:172.21.15.7:6801/1193301030] [v2:172.21.15.7:6802/1193301030,v1:172.21.15.7:6803/1193301030] exists,up d2261455-c91b-414b-925f-388f0e094dbe 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6810/2283562461,v1:172.21.15.189:6811/2283562461] [v2:172.21.15.189:6812/2283562461,v1:172.21.15.189:6813/2283562461] exists,up 4088d23b-0d06-4415-9eba-6fb64355746c 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6818/2212769626,v1:172.21.15.189:6819/2212769626] [v2:172.21.15.189:6820/2212769626,v1:172.21.15.189:6821/2212769626] exists,up 5e27cb53-ecce-4061-bc5a-be599bf18cc4 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6826/3197231389,v1:172.21.15.189:6827/3197231389] [v2:172.21.15.189:6828/3197231389,v1:172.21.15.189:6829/3197231389] exists,up ceb9a1f1-f54c-48d5-91d8-2287cac96ceb 2024-09-14T06:59:26.714 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/2162534312,v1:172.21.15.189:6801/2162534312] [v2:172.21.15.189:6802/2162534312,v1:172.21.15.189:6803/2162534312] exists,up 85c59cd6-6e2b-4cc0-852d-d1f075a8826d 2024-09-14T06:59:26.725 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6816/2081065366,v1:172.21.15.7:6817/2081065366] [v2:172.21.15.7:6818/2081065366,v1:172.21.15.7:6819/2081065366] exists,up c036038d-cffb-4584-8851-69604a31ee40', 'osd.1 up in weight 1 up_from 15 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6804/1688050909,v1:172.21.15.7:6806/1688050909] [v2:172.21.15.7:6808/1688050909,v1:172.21.15.7:6810/1688050909] exists,up 1c80080f-932a-4957-b64b-f30c4bdb9340', 'osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6824/1872259061,v1:172.21.15.7:6825/1872259061] [v2:172.21.15.7:6826/1872259061,v1:172.21.15.7:6827/1872259061] exists,up f3544d11-8a0f-498c-a03c-5d34134c9773', 'osd.3 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6800/1193301030,v1:172.21.15.7:6801/1193301030] [v2:172.21.15.7:6802/1193301030,v1:172.21.15.7:6803/1193301030] exists,up d2261455-c91b-414b-925f-388f0e094dbe', 'osd.4 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6810/2283562461,v1:172.21.15.189:6811/2283562461] [v2:172.21.15.189:6812/2283562461,v1:172.21.15.189:6813/2283562461] exists,up 4088d23b-0d06-4415-9eba-6fb64355746c', 'osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6818/2212769626,v1:172.21.15.189:6819/2212769626] [v2:172.21.15.189:6820/2212769626,v1:172.21.15.189:6821/2212769626] exists,up 5e27cb53-ecce-4061-bc5a-be599bf18cc4', 'osd.6 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6826/3197231389,v1:172.21.15.189:6827/3197231389] [v2:172.21.15.189:6828/3197231389,v1:172.21.15.189:6829/3197231389] exists,up ceb9a1f1-f54c-48d5-91d8-2287cac96ceb', 'osd.7 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/2162534312,v1:172.21.15.189:6801/2162534312] [v2:172.21.15.189:6802/2162534312,v1:172.21.15.189:6803/2162534312] exists,up 85c59cd6-6e2b-4cc0-852d-d1f075a8826d'] 2024-09-14T06:59:26.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-14T06:59:26.827 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mon.a", 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster": "ceph", 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "admin_socket_mode": "", 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_client_required": "cephx, none", 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_cluster_required": "cephx", 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_debug": "true", 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-14T06:59:26.828 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_service_required": "cephx", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_supported": "cephx", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio": "true", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_poll_ms": "250", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_reap_max": "16", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_async_discard_threads": "0", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_block_size": "4096", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio": "false", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-14T06:59:26.829 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_inflight_ios": "false", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_enable_discard": "false", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_flock_retry": "3", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_inject_crash": "0", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring": "false", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring_hipri": "false", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-14T06:59:26.830 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_type": "", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_alloc_size": "1048576", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_allocator": "hybrid", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_buffered_io": "true", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_for_zeros": "false", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_compact_log_sync": "false", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-14T06:59:26.831 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_max_log_runway": "4194304", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_max_prefetch": "1048576", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_min_flush_size": "524288", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_min_log_runway": "1048576", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_replay_recovery": "false", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_sync_write": "false", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_allocation_from_file": "true", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_allocator": "hybrid", 2024-09-14T06:59:26.832 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_async_db_compaction": "true", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_multi": "true", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_create": "true", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_create": "false", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_path": "", 2024-09-14T06:59:26.833 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_size": "0", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_path": "", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_preallocate_file": "false", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_size": "107374182400", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_create": "false", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_path": "", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_size": "100663296", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs": "true", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T06:59:26.834 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_autotune": "true", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size": "0", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-14T06:59:26.835 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_type": "2q", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_clone_cow": "true", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_mode": "none", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_csum_type": "crc32c", 2024-09-14T06:59:26.836 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_freelist": "false", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-14T06:59:26.837 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_small_allocations": "0", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_default_buffered_read": "true", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_default_buffered_write": "false", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-14T06:59:26.838 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fail_eio": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mount": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_umount": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-14T06:59:26.839 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_ignore_data_csum": "false", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_op_age": "5.000000", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_alloc_size": "0", 2024-09-14T06:59:26.840 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size": "0", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_deferred_txc": "32", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size": "0", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_nid_prealloc": "1024", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_retry_disk_reads": "3", 2024-09-14T06:59:26.841 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_cf": "true", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_mem": "512", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-14T06:59:26.842 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_tracing": "false", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-14T06:59:26.843 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_write_v2": "false", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_write_v2_random": "false", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_zero_block_detection": "false", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-14T06:59:26.844 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_cluster_require_signatures": "false", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_cluster_require_version": "2", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_require_signatures": "false", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_require_version": "2", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_service_require_signatures": "false", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_service_require_version": "2", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_sign_messages": "true", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "chdir": "", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_acl_type": "", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_asio_thread_count": "2", 2024-09-14T06:59:26.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_cache_mid": "0.750000", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_cache_size": "16384", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_caps_release_delay": "5", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_check_pool_perm": "true", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_force_sync_read": "false", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_getattr_caps": "false", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_inject_features": "", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_inject_tick_delay": "0", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_die_on_failed_remount": "false", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_dirsize_rbytes": "true", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_force_lazyio": "false", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_fs": "", 2024-09-14T06:59:26.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_inject_release_failure": "false", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_max_inline_size": "4096", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mds_namespace": "", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_metadata": "", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_gid": "-1", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_timeout": "300", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_uid": "-1", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mountpoint": "/", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_notify_timeout": "10", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc": "true", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_dirty": "104857600", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_objects": "1000", 2024-09-14T06:59:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_size": "209715200", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_target_dirty": "8388608", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_permissions": "true", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_quota": "true", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_quota_df": "true", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_max_bytes": "0", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_max_periods": "4", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_min": "131072", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_reconnect_stale": "false", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_shutdown_timeout": "30", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_snapdir": ".snap", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_tick_interval": "1", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_trace": "", 2024-09-14T06:59:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_try_dentry_invalidate": "false", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_use_faked_inos": "false", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_use_random_mds": "false", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog": "false", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog_port": "12201", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_monitors": "default=true", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog": "false", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog_level": "info", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "-", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_network": "", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_network_interface": "", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_isal": "false", 2024-09-14T06:59:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_level": "5", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_winsize": "-15", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zstd_level": "1", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_alien_op_num_threads": "6", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_stat_interval": "0", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_seastar_cpu_cores": "", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_seastar_num_threads": "0", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location_hook": "", 2024-09-14T06:59:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location_hook_timeout": "10", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "daemonize": "false", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "daos_pool": "tank", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_alienstore": "0/5", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_allow_any_pool_priority": "false", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asok": "1/5", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asok_assert_abort": "false", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asserts_on_shutdown": "true", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_auth": "20/20", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bdev": "1/3", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bluefs": "1/5", 2024-09-14T06:59:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bluestore": "1/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_buffer": "0/1", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ceph_dedup": "0/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ceph_exporter": "1/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cephfs_mirror": "0/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cephsqlite": "0/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_client": "0/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_compressor": "1/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_context": "0/1", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crimson_interrupt": "0/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crush": "1/1", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crypto": "1/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cyanstore": "0/5", 2024-09-14T06:59:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_deliberately_leak_memory": "false", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_disable_randomized_ping": "false", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_dpdk": "1/5", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_eventtrace": "1/5", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_filer": "0/1", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_filestore": "1/3", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_finisher": "1/1", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_fuse": "1/5", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_heartbeat_testing_span": "0", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_heartbeatmap": "1/5", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_javaclient": "1/5", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_journal": "1/3", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_journaler": "0/5", 2024-09-14T06:59:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_kstore": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_lockdep": "0/1", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mclock": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_balancer": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_locker": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_log": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_log_expire": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_migrator": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_quiesce": "3/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_memstore": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mgr": "2/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mgrc": "1/5", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mon": "20/20", 2024-09-14T06:59:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_monc": "0/10", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ms": "1/1", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_none": "0/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objclass": "0/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objectcacher": "0/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objecter": "0/1", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_optracker": "0/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_osd": "1/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_paxos": "20/20", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_perfcounter": "1/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_prioritycache": "1/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rados": "0/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd": "0/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_mirror": "0/5", 2024-09-14T06:59:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_pwl": "0/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_replay": "0/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_refs": "0/0", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_reserver": "1/1", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw": "1/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_access": "1/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_datacache": "1/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_dbstore": "1/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_flight": "1/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_notification": "1/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_sync": "1/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rocksdb": "4/5", 2024-09-14T06:59:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_backref": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_cache": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_cleaner": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_device": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_epm": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_journal": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_lba": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_odata": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_omap": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_onode": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_t": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_tm": "0/5", 2024-09-14T06:59:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_striper": "0/1", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_test": "0/5", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_throttle": "1/1", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_timer": "0/1", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_tp": "0/5", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_trace": "1/5", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "device_failure_prediction_mode": "none", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_graylog": "false", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_journald": "false", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_stderr": "true", 2024-09-14T06:59:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_syslog": "false", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event_tracing": "false", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_addr": "0.0.0.0", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_cert_file": "", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_http_port": "9926", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_key_file": "", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_prio_limit": "5", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_sort_metrics": "true", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_stats_period": "5", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "fake_statfs_for_testing": "0", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "fatal_signal_handlers": "true", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "filer_max_purge_ops": "10", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "filer_max_truncate_ops": "128", 2024-09-14T06:59:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_apply_finisher_threads": "1", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_blackhole": "false", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_btrfs_clone_range": "true", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_btrfs_snap": "true", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_caller_concurrency": "10", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_collect_device_partition_information": "true", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_commit_timeout": "600.000000", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_inject_read_err": "false", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_omap_check": "false", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_verify_split": "false", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_dump_file": "", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fadvise": "true", 2024-09-14T06:59:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fail_eio": "true", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fd_cache_shards": "16", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fd_cache_size": "128", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fiemap": "false", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fiemap_threshold": "4096", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_inject_stall": "0", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_parallel": "false", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_trailing": "false", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_writeahead": "false", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_kill_at": "0", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-14T06:59:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs": "0", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size": "0", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-14T06:59:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_merge_threshold": "-10", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_odsync_write": "false", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_backend": "rocksdb", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_backend_path": "", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_thread_timeout": "60", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_threads": "2", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_punch_hole": "false", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-14T06:59:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_ops": "50", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_seek_data_hole": "false", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_sloppy_crc": "false", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_splice": "false", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_split_multiple": "2", 2024-09-14T06:59:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_split_rand_factor": "20", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_update_to": "1000", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_enable": "true", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-14T06:59:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_xfs_extsize": "false", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_zfs_snap": "false", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fio_dir": "/tmp/fio", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_allow_other": "true", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_atomic_o_trunc": "true", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_big_writes": "true", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_debug": "false", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_default_permissions": "false", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_disable_pagecache": "false", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_max_write": "0", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_multithreaded": "true", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_require_active_mds": "true", 2024-09-14T06:59:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_set_user_groups": "true", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_move": "true", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_read": "true", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_write": "true", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_use_invalidate_cb": "true", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "gss_target_name": "ceph", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_file": "", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_inject_failure": "0", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_interval": "5", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "host": "smithi007", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-14T06:59:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "inject_early_sigterm": "false", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "jaeger_agent_port": "6799", 2024-09-14T06:59:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "jaeger_tracing_enable": "false", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_aio": "true", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_align_min_size": "65536", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_block_align": "true", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_block_size": "4096", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_dio": "true", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_discard": "false", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_force_aio": "false", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_ignore_corruption": "false", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_max_write_bytes": "10485760", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_max_write_entries": "100", 2024-09-14T06:59:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_replay_from": "0", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_write_header_frequency": "0", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_zero_on_create": "false", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_prefetch_periods": "10", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_prezero_periods": "5", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_write_head_interval": "15", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "keyfile": "", 2024-09-14T06:59:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_backend": "rocksdb", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_default_stripe_size": "65536", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_fsck_on_mount": "false", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_max_bytes": "67108864", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_max_ops": "512", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_nid_prealloc": "1024", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_onode_map_size": "1024", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_sync_submit_transaction": "false", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_sync_transaction": "false", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "librados_thread_count": "2", 2024-09-14T06:59:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "lockdep": "false", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "lockdep_force_backtrace": "false", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_coarse_timestamps": "true", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_flush_on_exit": "false", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_graylog_host": "127.0.0.1", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_graylog_port": "12201", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_max_new": "1000", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_max_recent": "10000", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_stderr_prefix": "", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_stop_at_utilization": "0.970000", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": "true", 2024-09-14T06:59:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_graylog": "false", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_journald": "false", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_stderr": "false", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_syslog": "false", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "max_rotating_auth_attempts": "10", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_action_on_write_error": "1", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_alternate_name_max": "8192", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_asio_thread_count": "2", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_export_pin": "true", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_dirs": "true", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-14T06:59:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_interval": "5", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_interval": "10", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_max": "-1", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_max_until": "-1", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_merge_size": "50", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_midchunk": "0.300000", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_min_start": "0.200000", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_minchunk": "0.001000", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_mode": "0", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_need_max": "1.200000", 2024-09-14T06:59:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_need_min": "0.800000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_overload_epochs": "2", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_bits": "3", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_size": "10000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_target_decay": "10.000000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_grace": "15.000000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_interval": "4.000000", 2024-09-14T06:59:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_mid": "0.700000", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_delay": "0", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_release_free_interval": "10", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_reservation": "0.050000", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_interval": "1", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_threshold": "262144", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-14T06:59:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_prealloc_inos": "1000", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_connect_bootstrapping": "false", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_damage_table_max_entries": "10000", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_auth_pins": "false", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_frag": "false", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_scatterstat": "false", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_subtrees": "false", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_decay_halflife": "5.000000", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_default_dir_hash": "2", 2024-09-14T06:59:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_defer_session_stale": "true", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_deny_all_reconnect": "false", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_keys_per_op": "16384", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_max_commit_size": "10", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_max_entries": "0", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_prefetch": "true", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_on_map": "false", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_early_reply": "true", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_enable_op_tracker": "true", 2024-09-14T06:59:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_enforce_unique_name": "true", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_random": "true", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_health_summarize_threshold": "10", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-14T06:59:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_health_dummy": "false", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_migrator_session_race": "false", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_join_fs": "", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_journal_format": "1", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_create_at": "0", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_dirfrag_at": "0", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_export_at": "0", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_import_at": "0", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_at": "0", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_expire_at": "0", 2024-09-14T06:59:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_replay_at": "0", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_link_at": "0", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_mdstable_at": "0", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_openc_at": "0", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_rename_at": "0", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_shutdown_at": "0", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_event_large_threshold": "524288", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_events_per_segment": "1024", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_max_events": "-1", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_max_segments": "128", 2024-09-14T06:59:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_pause": "false", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_segment_size": "0", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_threshold": "128", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_warn_factor": "2.000000", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_caps_per_client": "1048576", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_completed_flushes": "100000", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_completed_requests": "100000", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_export_size": "20971520", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_file_recover": "32", 2024-09-14T06:59:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_files": "64", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_ops": "8192", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_snaps_per_dir": "100", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_metrics_update_interval": "2", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_min_caps_per_client": "100", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_min_caps_working_set": "10000", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_numa_node": "-1", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_complaint_time": "30.000000", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_duration": "600", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_size": "20", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_slow_op_size": "20", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-14T06:59:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_log_threshold": "5", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_ping_grace": "15", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_ping_interval": "5", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_caps": "30000", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_warning_threshold": "262144", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_replay_interval": "1.000000", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-14T06:59:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_root_ino_gid": "0", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_root_ino_uid": "0", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_scrub_stats_review_period": "1", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-14T06:59:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_shutdown_check": "0", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_skip_ino": "0", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_max_uid": "4294967294", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_min_uid": "0", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_rstat": "false", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_standby_replay_damaged": "false", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_symlink_recovery": "true", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_thrash_exports": "0", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_thrash_fragments": "0", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_tick_interval": "5.000000", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_valgrind_exit": "false", 2024-09-14T06:59:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_verify_backtrace": "1", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_verify_scatter": "false", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_wipe_sessions": "false", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool_debug": "false", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_device_bytes": "1073741824", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_page_set": "false", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_page_size": "65536", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_bytes": "134217728", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_messages": "512", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-14T06:59:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_disabled_modules": "", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_enable_op_tracker": "true", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_max_pg_creating": "1024", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_max_pg_num_change": "128", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mds_bytes": "134217728", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mds_messages": "128", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mon_bytes": "134217728", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mon_messages": "128", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-14T06:59:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_duration": "600", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_size": "20", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_log_threshold": "5", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_osd_bytes": "536870912", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_osd_messages": "8192", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_pool": "true", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_standby_modules": "true", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_stats_period": "5", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_stats_threshold": "5", 2024-09-14T06:59:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_tick_period": "2", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_allow_pool_delete": "true", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_allow_pool_size_one": "true", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_auth_validate_all_caps": "true", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_bytes": "104857600", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_directed_command_retry": "5", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_parallel": "3", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_log_interval": "1.000000", 2024-09-14T06:59:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_ping_interval": "10.000000", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_target_rank": "-1", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_level": "debug", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_file": "true", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_journald": "false", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-14T06:59:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_bootstrap": "false", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_start": "false", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_trim": "true", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cpu_threads": "4", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_crush_min_required_version": "hammer", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_daemon_bytes": "419430400", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_avail_crit": "5", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_avail_warn": "5", 2024-09-14T06:59:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_size_warn": "16106127360", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_json": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_transactions": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_extra_checks": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_reef": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_squid": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-14T06:59:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_down_mkfs_grace": "120", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_down_uptime_grace": "60", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_election_default_strategy": "1", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_election_timeout": "5.000000", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ping_divisor": "2", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_enable_op_tracker": "true", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_fake_pool_delete": "false", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_force_quorum_join": "false", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_globalid_prealloc": "10000", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_detail_to_clog": "false", 2024-09-14T06:59:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_log_update_period": "5", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_max_detail": "50", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog": "true", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog_interval": "600", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_host": "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.189", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_host_override": "", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_initial_members": "", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease": "5.000000", 2024-09-14T06:59:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_full_interval": "50", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_max": "10000", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_max_summary": "50", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_log_epochs": "500", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_osd": "10000", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_pg_per_osd": "10000", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_pool_pg_num": "65536", 2024-09-14T06:59:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_force_trim_to": "0", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_skip_sanity": "false", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_memory_autotune": "true", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_memory_target": "2147483648", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_beacon_grace": "30", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_digest_period": "5", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_inactive_grace": "60", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_min_osdmap_epochs": "500", 2024-09-14T06:59:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_complaint_time": "30", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_duration": "600", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_size": "20", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_slow_op_size": "20", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_log_threshold": "5", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_in": "false", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-14T06:59:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_cache_size": "500", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_down_out_interval": "600", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_force_trim_to": "0", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-14T06:59:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_down_reporters": "2", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_report_timeout": "900", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-14T06:59:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_stuck_threshold": "60", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_probe_timeout": "2.000000", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_max_change": "0.050000", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_max_osds": "4", 2024-09-14T06:59:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_interval": "86400", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_max_keys": "100", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_timeout": "300", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_session_timeout": "300", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_smart_report_timeout": "5", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stat_smooth_intervals": "6", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_pool_min_size": "2", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_pool_size": "4", 2024-09-14T06:59:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_debug": "false", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_provider_kill_at": "0", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_requester_kill_at": "0", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_timeout": "60.000000", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_target_pg_per_osd": "100", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_tick_interval": "5", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_timecheck_interval": "300.000000", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_use_min_delay_socket": "false", 2024-09-14T06:59:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_older_version_delay": "604800", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_misplaced": "false", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_older_version": "true", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-14T06:59:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": "", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_tracing_enabled": "false", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_op_threads": "3", 2024-09-14T06:59:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_cm": "false", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_device_name": "", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_dscp": "96", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_local_gid": "", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_port_num": "1", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_sl": "3", 2024-09-14T06:59:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_support_srq": "true", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_type": "ib", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_reap_threshold": "5", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_before_connect": "false", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_ipv4": "true", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_ipv6": "false", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_msgr1": "true", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_msgr2": "true", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_port_max": "7568", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_port_min": "6800", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_retry_count": "3", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_retry_delay": "5", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_client": "false", 2024-09-14T06:59:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mds": "false", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mgr": "false", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mon": "false", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_osd": "false", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_client_mode": "crc secure", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_cluster_mode": "crc secure", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_cluster_type": "", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_compress_secure": "false", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_connection_idle_timeout": "900", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_connection_ready_timeout": "10", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_crc_data": "true", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_crc_header": "true", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_bad_msg": "false", 2024-09-14T06:59:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_bug": "true", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_old_message": "true", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_skipped_message": "false", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_coremask": "0xF", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_enable_tso": "true", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hugepages": "", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-14T06:59:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_lro": "true", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_memory_channel": "4", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_pmd": "", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_port_id": "0", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dump_on_send": "false", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_initial_backoff": "0.200000", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_max": "1.000000", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_type": "", 2024-09-14T06:59:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_network_congestion": "0", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_socket_failures": "5000", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_learn_addr_from_peer": "true", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_max_accept_failures": "4", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_max_backoff": "15.000000", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_client_mode": "secure crc", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_service_mode": "secure crc", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compress_min_size": "1024", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compress_mode": "none", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_pq_min_cost": "65536", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_public_type": "", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_service_mode": "crc secure", 2024-09-14T06:59:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_listen_backlog": "512", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_nodelay": "true", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_rcvbuf": "0", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_type": "async+posix", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "no_config_file": "false", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_completion_locks_per_session": "32", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inflight_ops": "1024", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-14T06:59:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_tick_interval": "5.000000", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_timeout": "10.000000", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "objectstore_blackhole": "false", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "openssl_engine_opts": "", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_delay_time": "5.000000", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_hist_halflife": "1000", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_max_low_ops": "2", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_max_ops": "4", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-14T06:59:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_slop": "0.020000", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_asio_thread_count": "2", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_async_recovery_min_cost": "100", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_scan_max": "512", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_scan_min": "64", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_degraded": "false", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_peering": "false", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_unfound": "true", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_beacon_report_interval": "300", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_duration": "30", 2024-09-14T06:59:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_max_block_size": "67108864", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_blkin_trace_all": "false", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_check_for_log_corruption": "false", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-14T06:59:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_update_on_start": "true", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_message_cap": "256", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_message_size_cap": "524288000", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_op_priority": "63", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_watch_timeout": "30", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_command_max_records": "256", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_compact_on_start": "false", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_update_on_start": "true", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_update_weight_set": "true", 2024-09-14T06:59:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_feed_pullee": "-1", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_misdirected_ops": "false", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_no_acting_change": "false", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_no_purge_strays": "false", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_op_order": "false", 2024-09-14T06:59:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_shutdown": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_trim_objects": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_snaps": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_keys": "1024", 2024-09-14T06:59:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_stride": "524288", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_default_notify_timeout": "30", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep": "0.000000", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_discard_disconnected_ops": "true", 2024-09-14T06:59:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_ec_partial_reads": "true", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_enable_op_tracker": "true", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_info": "true", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown": "true", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-14T06:59:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_function_tracing": "false", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_grace": "20", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_interval": "6", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_peers": "10", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_size": "2000", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_stale": "600", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_max_size": "100000", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_min_size": "1000", 2024-09-14T06:59:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal_size": "5120", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_kill_backfill_at": "0", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_cache_size": "50", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_dedup": "true", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_message_max": "40", 2024-09-14T06:59:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_share_max_epochs": "40", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_attr_name_len": "100", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_attr_size": "0", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_backfills": "1", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_markdown_count": "5", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_markdown_period": "600", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_name_len": "2048", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_namespace_len": "256", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_size": "134217728", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-14T06:59:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_blocked_by": "16", 2024-09-14T06:59:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_log_entries": "10000", 2024-09-14T06:59:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-14T06:59:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pgls": "1024", 2024-09-14T06:59:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_push_cost": "8388608", 2024-09-14T06:59:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_push_objects": "10", 2024-09-14T06:59:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_scrubs": "3", 2024-09-14T06:59:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-14T06:59:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_trimming_pgs": "2", 2024-09-14T06:59:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_write_op_reply_len": "64", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_write_size": "90", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_profile": "balanced", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-14T06:59:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_base": "805306368", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_cache_min": "134217728", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target": "4294967296", 2024-09-14T06:59:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target_autotune": "false", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_min_pg_log_entries": "250", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_min_recovery_priority": "0", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_report_interval": "5", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_num_cache_shards": "32", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_num_op_tracker_shard": "32", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_auto_affinity": "true", 2024-09-14T06:59:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_node": "-1", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_prefer_iface": "true", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objecter_finishers": "1", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore": "bluestore", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_fuse": "false", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_tracing": "false", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_complaint_time": "30.000000", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_duration": "600", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_size": "20", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_slow_op_size": "20", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_log_threshold": "5", 2024-09-14T06:59:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards": "0", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards_hdd": "5", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards_ssd": "8", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_pq_min_cost": "65536", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_queue_cut_off": "high", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_thread_timeout": "15", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_open_classes_on_start": "true", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_os_flags": "0", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_peering_op_priority": "255", 2024-09-14T06:59:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_delete_cost": "1048576", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_delete_priority": "5", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_trim_max": "10000", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_trim_min": "100", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-14T06:59:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_crimson": "false", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flags": "0", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_min_size": "0", 2024-09-14T06:59:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pg_num": "32", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pgp_num": "0", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_read_ratio": "70", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_size": "2", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_type": "replicated", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_push_per_object_cost": "1000", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recover_clone_overlap": "true", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_cost": "20971520", 2024-09-14T06:59:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active": "0", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_single_start": "1", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_op_priority": "3", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_priority": "5", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep": "0.000000", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-14T06:59:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_repair_during_recovery": "false", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_requested_scrub_priority": "120", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_auto_repair": "false", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_begin_hour": "0", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_begin_week_day": "0", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_chunk_max": "25", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_chunk_min": "5", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_cost": "52428800", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_during_recovery": "false", 2024-09-14T06:59:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_end_hour": "0", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_end_week_day": "0", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_event_cost": "4096", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_invalid_stats": "true", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_max_preemptions": "5", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_priority": "5", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_sleep": "0.000000", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-14T06:59:26.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_skip_data_digest": "false", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_smart_report_timeout": "5", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_cost": "1048576", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_priority": "5", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_target_transaction_size": "30", 2024-09-14T06:59:26.936 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tracing": "false", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_use_stale_snap": "false", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osdc_blkin_trace_all": "false", 2024-09-14T06:59:26.937 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_kill_at": "0", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_max_join_drift": "10", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_min": "500", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_min_wait": "0.050000", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_propose_interval": "1.000000", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_max": "500", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_min": "250", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_stash_full_interval": "25", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_trim_max": "500", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_trim_min": "250", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "perf": "true", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-14T06:59:26.938 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "-", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrv": "", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "public_bind_addr": "-", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "public_network": "", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "public_network_interface": "", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_busy_polling": "false", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_enabled": "false", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_session_max_number": "256", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_mon_op_timeout": "0", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_osd_op_timeout": "0", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_tracing": "false", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_atime_update_interval": "60", 2024-09-14T06:59:26.939 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_balance_parent_reads": "false", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_balance_snap_reads": "false", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blkin_trace_all": "false", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache": "true", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_policy": "writearound", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_size": "33554432", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-14T06:59:26.940 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_clone_copy_on_read": "false", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_compression_hint": "none", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_concurrent_management_ops": "10", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_clone_format": "auto", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_data_pool": "", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_features": "61", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_format": "2", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_map_options": "", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_order": "22", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_pool": "rbd", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_stripe_count": "0", 2024-09-14T06:59:26.941 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_stripe_unit": "0", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_enable_alloc_hint": "true", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_io_scheduler": "simple", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-14T06:59:26.942 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_order": "24", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_pool": "", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_splay_width": "4", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_localize_parent_reads": "false", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_localize_snap_reads": "false", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-14T06:59:26.943 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-14T06:59:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mtime_update_interval": "60", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_non_blocking_aio": "true", 2024-09-14T06:59:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_op_thread_timeout": "60", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_op_threads": "1", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_parent_cache_enabled": "false", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_plugins": "", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_burst": "0", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_limit": "0", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_exclude_ops": "0", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_burst": "0", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_limit": "0", 2024-09-14T06:59:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-14T06:59:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_read_from_replica_policy": "default", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_skip_partial_discard": "true", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_tracing": "false", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_validate_names": "true", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_validate_pool": "true", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "restapi_base_url": "", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "restapi_log_level": "", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-14T06:59:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_acl_grants_max_num": "100", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_admin_entry": "admin", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_asio_assert_yielding": "false", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_backend_store": "rados", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_barbican_url": "", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_beast_enable_async": "true", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_counters_cache": "false", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-14T06:59:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_enabled": "true", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_expiry_interval": "900", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_lru_size": "25000", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_config_store": "rados", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_content_length_compat": "false", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_copy_obj_progress": "true", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cors_rules_max_num": "100", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cross_domain_policy": "", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_addr": "", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-14T06:59:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_password": "", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_username": "", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_require_ssl": "true", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-14T06:59:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_addr": "", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_auth": "token", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_namespace": "", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_prefix": "", 2024-09-14T06:59:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_token_file": "", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_buffersize": "524288", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_low_speed_time": "300", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-14T06:59:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-14T06:59:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_changes_size": "1000", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_num_shards": "128", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_window": "30", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-14T06:59:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_disable_s3select": "false", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-14T06:59:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dns_name": "", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dns_s3website_name": "", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding": "true", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_gc_threads": "true", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_lc_threads": "true", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_mdsearch": "true", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_ops_log": "false", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_quota_threads": "true", 2024-09-14T06:59:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_static_website": "false", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_usage_log": "false", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enforce_swift_acls": "true", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_exit_timeout_secs": "120", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_expose_bucket": "false", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_extended_http_attrs": "", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_filter": "none", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_frontends": "beast port=7480", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_deferred": "50", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_objs": "32", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-14T06:59:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_processor_period": "3600", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_init_timeout": "300", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_connection_idle": "300", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-14T06:59:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_domain": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_password": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_password_path": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_project": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_tenant": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_token": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_token_path": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_user": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_domain": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_password": "", 2024-09-14T06:59:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_project": "", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_user": "", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_url": "", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_debug_interval": "-1", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_lock_max_time": "90", 2024-09-14T06:59:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_objs": "32", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_rules": "1000", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_worker": "3", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_thread_delay": "0", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_dnattr": "uid", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_searchfilter": "", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_uri": "", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-14T06:59:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_http_headers": "", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_object_name_utc": "false", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attr_name_len": "0", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attr_size": "0", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_chunk_size": "4194304", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-14T06:59:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_listing_results": "1000", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_notify_retries": "10", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_put_param_size": "1048576", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_put_size": "5368709120", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_slo_entries": "1000", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_md_log_max_shards": "64", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_mp_lock_max_time": "600", 2024-09-14T06:59:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_max_gc": "300", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-14T06:59:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_num_async_rados_threads": "32", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_num_control_oids": "8", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_numa_node": "-1", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_chunk_size": "100", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_gc_interval": "600", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-14T06:59:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_thread_timeout": "600", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_tracing": "false", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_token": "", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_url": "", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_verify_ssl": "true", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_rados": "false", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_socket_path": "", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-14T06:59:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_push_interval": "2.000000", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_lanes": "3", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_partitions": "3", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_print_continue": "true", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-14T06:59:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_tracing": "false", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm": "", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm_id": "", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_region": "", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-14T06:59:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_request_uri": "", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_batch_size": "64", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_max_aio": "128", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_num_logs": "16", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_thread_interval": "600", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_resolve_cname": "false", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_run_sync_thread": "true", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-14T06:59:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_scheduler_type": "throttler", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_script_uri": "", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_service_provider_name": "", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_client_id": "", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_client_secret": "", 2024-09-14T06:59:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_entry": "sts", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_key": "sts", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_min_session_duration": "900", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_token_introspection_url": "", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_account_in_url": "false", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_auth_entry": "auth", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_auth_url": "", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_custom_header": "", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_need_stats": "true", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_tenant_name": "", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_token_expiration": "86400", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_url": "", 2024-09-14T06:59:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_url_prefix": "swift", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_lease_period": "120", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-14T06:59:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_thread_pool_size": "512", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_comment": "", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_createby": "", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_encoding": "", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_flag": "false", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_origin": "", 2024-09-14T06:59:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_tracker": "", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_trust_forwarded_https": "false", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_max_shards": "32", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_max_user_shards": "1", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_use_opa_authz": "false", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_counters_cache": "false", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_max_buckets": "1000", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_policies_max_num": "100", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-14T06:59:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_unique_email": "true", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_verify_ssl": "true", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone": "", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone_id": "", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup": "", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup_id": "", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgwlc_auto_session_clear": "true", 2024-09-14T06:59:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_block_size": "4096", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_size": "536870912", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-14T06:59:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_index_type": "binary_search", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_partition_filters": "false", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_perf": "false", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "run_dir": "/var/run/ceph", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_block_create": "true", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_cache_lru_size": "67108864", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_default_max_object_size": "16777216", 2024-09-14T06:59:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_device_size": "53687091200", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_full_integrity_check": "false", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_capacity": "16", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_main_device_type": "SSD", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-14T06:59:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_segment_size": "67108864", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "set_keepcaps": "false", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "setgroup": "", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "setuser": "", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "setuser_match_path": "", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "thp": "false", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "threadpool_default_timeout": "60", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "throttler_perf_counter": "true", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tmp_dir": "/tmp", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "uadk_compressor_enabled": "false", 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-14T06:59:26.980 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:26.982 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-14T06:59:27.100 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a: {} 2024-09-14T06:59:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:59:27.110 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b: {} 2024-09-14T06:59:27.110 INFO:teuthology.orchestra.run.smithi007.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:59:27.119 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c: {} 2024-09-14T06:59:27.119 INFO:teuthology.orchestra.run.smithi007.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:59:27.130 DEBUG:teuthology.orchestra.run.smithi189:> type ceph-objectstore-tool 2024-09-14T06:59:27.148 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T06:59:27.148 DEBUG:teuthology.orchestra.run.smithi007:> type ceph-objectstore-tool 2024-09-14T06:59:27.163 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T06:59:27.163 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-14T06:59:27.173 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-14T06:59:27.173 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-14T06:59:27.173 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:59:27.173 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:59:27.174 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-14T06:59:27.174 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:59:27.174 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:59:27.174 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-14T06:59:27.174 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-14T06:59:27.175 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-14T06:59:27.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:59:27.175 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-14T06:59:27.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:59:27.176 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-14T06:59:27.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:59:27.176 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-14T06:59:27.204 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:27.203+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:27.239 INFO:teuthology.orchestra.run.smithi007.stdout:mapped writes DISABLED 2024-09-14T06:59:27.239 INFO:teuthology.orchestra.run.smithi007.stdout:mapped reads DISABLED 2024-09-14T06:59:27.239 INFO:teuthology.orchestra.run.smithi007.stdout:Seed set to 3737 2024-09-14T06:59:27.277 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:27.275+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:27.302 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:27.302 INFO:teuthology.orchestra.run.smithi189.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:59:27.302 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:27.319 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:27.319 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:27.319 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:27.319 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:27.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:59:27.341 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:27.360 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:27.360 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:27.377 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:27.376+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:27.378 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:27.378 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:27.394 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:27.412 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:27.412 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:27.429 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:27.441 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:27.441 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:27.446 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:27.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:59:27.464 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:27.465 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:27.517 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:27.515+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:27.572 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:27.572 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:27.579 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:27.578+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:27.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:59:27.681 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:27.680+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:27.728 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:27.728 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:27.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:59:27.781 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:27.780+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T06:59:27.850 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:27.851 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:27.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:59:27.883 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:27.882+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:27.966 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:27.966 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:27.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:59:27.983 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:27.983+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:28.083 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:28.083 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:28.084 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:28.082+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:28.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:59:28.185 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:28.184+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:28.239 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:28.239 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:28.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:59:28.285 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:28.285+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:28.354 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:28.355 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:28.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:59:28.407 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:28.405+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:28.471 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:28.471 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:28.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:59:28.521 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:28.520+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:28.589 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:28.587+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:28.628 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:28.628 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:28.636 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:59:28.689 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:28.689+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:28.783 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:28.783 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:28.790 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:28.789+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:28.792 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:59:28.892 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:28.891+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:28.908 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:28.909 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:28.917 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:59:28.993 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:28.992+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:29.021 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:29.021 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:29.029 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:59:29.094 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:29.093+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:29.134 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:29.134 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:29.143 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:59:29.194 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:29.193+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:29.286 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:29.286 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:29.295 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:59:29.296 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:29.294+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:29.396 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:29.395+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:29.439 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:29.439 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:29.447 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:59:29.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:59:29.521 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:29.519+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:29.591 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:29.591 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:29.597 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:29.596+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:29.600 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:59:29.614 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:29.615 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:29.632 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:29.632 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:29.648 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:29.648 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:29.664 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:29.664 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:29.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:29.698 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:29.699 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:29.699 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:29.698+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:29.703 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:29.704 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:29.713 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:59:29.715 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:29.715 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:29.732 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:29.732 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:29.799 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:29.799+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:29.809 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:59:29.857 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:29.857 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:29.857 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:29.857 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:29.857 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:29.865 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:59:29.906 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:29.905+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:30.001 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:30.001+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:30.008 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:30.008 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:30.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:30.008 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:30.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.008 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.2441509869999994, 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T06:59:30.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T06:59:30.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:21.768479+0000", 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 8.2398207540000001, 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00080781400000000001, 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:30.013 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.768479+0000", 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.768479+0000", 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.768479+0000", 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7799999999999997e-07 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.768487+0000", 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2789999999999998e-06 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.768503+0000", 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5984000000000001e-05 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.768513+0000", 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6870000000000006e-06 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.768561+0000", 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8430000000000002e-05 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:30.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.768581+0000", 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9140999999999999e-05 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.769243+0000", 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00066210399999999995 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:21.769287+0000", 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3911e-05 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:30.017 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:30.018 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:30.018 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:30.018 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:30.019 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:59:30.105 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:30.104+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:30.165 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:30.165 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:30.173 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:59:30.203 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:30.202+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:30.304 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:30.303+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:30.317 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:30.317 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:30.317 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:30.317 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:30.317 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:30.325 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:59:30.405 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:30.404+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:30.429 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:30.429 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:30.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:30.429 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [] 2024-09-14T06:59:30.429 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:30.437 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:59:30.506 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:30.506+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:30.548 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:30.549 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:30.549 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:30.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:59:30.607 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:30.607+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:30.663 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:30.663 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:30.664 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:30.664 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:30.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:59:30.716 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:30.716+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:30.779 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:30.779 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:30.780 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [] 2024-09-14T06:59:30.780 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:30.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:59:30.799 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T06:59:30.819 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:30.818+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:30.910 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:30.909+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:30.912 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:30.913 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:30.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:59:31.011 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:31.010+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:31.030 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:31.030 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:31.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:59:31.122 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:31.122+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:31.189 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:31.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.453488934999999, 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T06:59:31.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T06:59:31.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T06:59:31.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414215+0000", 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.772990706000002, 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0062819, 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.197 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 46 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414215+0000", 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414215+0000", 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414215+0000", 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1300000000000001e-07 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414219+0000", 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0160000000000002e-06 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414227+0000", 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:59:31.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414235+0000", 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3860000000000007e-06 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418085+0000", 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00385054 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418135+0000", 2024-09-14T06:59:31.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9163999999999999e-05 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418212+0000", 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6926000000000002e-05 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419422+0000", 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001210123 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420383+0000", 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000961596 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420467+0000", 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3967000000000004e-05 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420497+0000", 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9331000000000001e-05 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414255+0000", 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.772950406, 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069405730000000002, 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 47 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414255+0000", 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414255+0000", 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414256+0000", 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-07 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414347+0000", 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1472999999999996e-05 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414357+0000", 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-14T06:59:31.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414364+0000", 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.584e-06 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418461+0000", 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040973720000000002 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418550+0000", 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8548999999999995e-05 2024-09-14T06:59:31.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418682+0000", 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013265600000000001 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420504+0000", 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018218150000000001 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421143+0000", 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063939300000000004 2024-09-14T06:59:31.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421174+0000", 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0624999999999999e-05 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421196+0000", 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1454000000000002e-05 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.208 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414378+0000", 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.772827705000001, 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078204360000000001, 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 48 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414378+0000", 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414378+0000", 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414378+0000", 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414383+0000", 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6110000000000004e-06 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414390+0000", 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5000000000000002e-06 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414398+0000", 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3119999999999996e-06 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419441+0000", 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050428770000000003 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419489+0000", 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7877999999999998e-05 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419562+0000", 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3547999999999994e-05 2024-09-14T06:59:31.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420507+0000", 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094460900000000001 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422088+0000", 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015811250000000001 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422152+0000", 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4238000000000004e-05 2024-09-14T06:59:31.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422198+0000", 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6091000000000003e-05 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414416+0000", 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.772789532000001, 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078677350000000007, 2024-09-14T06:59:31.214 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414416+0000", 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414416+0000", 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414416+0000", 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414421+0000", 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3680000000000004e-06 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414429+0000", 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9859999999999994e-06 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414435+0000", 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8089999999999998e-06 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419804+0000", 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005369497 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419905+0000", 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100521 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420051+0000", 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014670199999999999 2024-09-14T06:59:31.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422028+0000", 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019769380000000001 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422232+0000", 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020340700000000001 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422259+0000", 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422284+0000", 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4573e-05 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.220 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414454+0000", 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.772751471999999, 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079161430000000005, 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 50 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414454+0000", 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414454+0000", 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414455+0000", 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414574+0000", 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119368 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414581+0000", 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8950000000000001e-06 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414589+0000", 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2449999999999998e-06 2024-09-14T06:59:31.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420538+0000", 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0059490569999999998 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420648+0000", 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011021799999999999 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420765+0000", 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116749 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422290+0000", 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001525059 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422314+0000", 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3533999999999999e-05 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422344+0000", 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0334000000000001e-05 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422370+0000", 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6069000000000001e-05 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.226 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414598+0000", 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.772607359999999, 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080802319999999997, 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 51 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414598+0000", 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414598+0000", 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414598+0000", 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6100000000000001e-07 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414601+0000", 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414606+0000", 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1410000000000004e-06 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414609+0000", 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.982e-06 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421243+0000", 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006634175 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421321+0000", 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7647999999999999e-05 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421440+0000", 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011891 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422381+0000", 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094076700000000001 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422629+0000", 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000247972 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422658+0000", 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9152000000000001e-05 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422678+0000", 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0517e-05 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414640+0000", 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.772565392000001, 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081134169999999995, 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 52 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414640+0000", 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414640+0000", 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414640+0000", 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414643+0000", 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414650+0000", 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5259999999999999e-06 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414653+0000", 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0149999999999999e-06 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421706+0000", 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070536139999999997 2024-09-14T06:59:31.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421755+0000", 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8702999999999998e-05 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421839+0000", 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3554999999999995e-05 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422386+0000", 2024-09-14T06:59:31.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054785700000000001 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422706+0000", 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031942199999999998 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422734+0000", 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8411999999999998e-05 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422753+0000", 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120999999999999e-05 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:54 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.423043+0000", 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.764162151000001, 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054059049999999999, 2024-09-14T06:59:31.238 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54 2024-09-14T06:59:31.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423043+0000", 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423043+0000", 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423044+0000", 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423105+0000", 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1645999999999999e-05 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423112+0000", 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0839999999999997e-06 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423116+0000", 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423529+0000", 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041364 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423609+0000", 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0133e-05 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423745+0000", 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013610599999999999 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424644+0000", 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089827699999999995 2024-09-14T06:59:31.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428393+0000", 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037495390000000001 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428427+0000", 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3852e-05 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428449+0000", 2024-09-14T06:59:31.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2067000000000001e-05 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:55 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.423126+0000", 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.764079339999999, 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054169220000000002, 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.245 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 55 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423126+0000", 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423126+0000", 2024-09-14T06:59:31.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423126+0000", 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2599999999999998e-07 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423187+0000", 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0185000000000001e-05 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423192+0000", 2024-09-14T06:59:31.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4979999999999997e-06 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423196+0000", 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1649999999999999e-06 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424071+0000", 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087498000000000003 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424118+0000", 2024-09-14T06:59:31.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6912000000000001e-05 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424228+0000", 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110087 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.425125+0000", 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089658999999999995 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428484+0000", 2024-09-14T06:59:31.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00335907 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428517+0000", 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3311e-05 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428543+0000", 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5698e-05 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.250 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:56 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.423204+0000", 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.764001230000002, 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008228282, 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 56 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423204+0000", 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423204+0000", 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423205+0000", 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-07 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423243+0000", 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8618000000000002e-05 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423248+0000", 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423256+0000", 2024-09-14T06:59:31.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424676+0000", 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001420227 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424741+0000", 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4943000000000005e-05 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424820+0000", 2024-09-14T06:59:31.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8654999999999998e-05 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.425403+0000", 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058315400000000001 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.431305+0000", 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0059021509999999996 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.431377+0000", 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2078999999999999e-05 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.431432+0000", 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5275999999999997e-05 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718658+0000", 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.468547094000002, 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005725178, 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 71 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718658+0000", 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718658+0000", 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718659+0000", 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718663+0000", 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.245e-06 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718689+0000", 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5982e-05 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718696+0000", 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0219999999999999e-06 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.721646+0000", 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029494539999999998 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.721690+0000", 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3968000000000003e-05 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.721758+0000", 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8134999999999997e-05 2024-09-14T06:59:31.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722880+0000", 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011223540000000001 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724334+0000", 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00145371 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724362+0000", 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8158e-05 2024-09-14T06:59:31.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724383+0000", 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1574999999999999e-05 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718712+0000", 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.468493054, 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057869210000000004, 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 72 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718712+0000", 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718712+0000", 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718713+0000", 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718818+0000", 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105278 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718835+0000", 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6779999999999999e-05 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718844+0000", 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4989999999999994e-06 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722033+0000", 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031890360000000001 2024-09-14T06:59:31.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722077+0000", 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4919999999999997e-05 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722153+0000", 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5764000000000002e-05 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723304+0000", 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011504379999999999 2024-09-14T06:59:31.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724430+0000", 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011259760000000001 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724457+0000", 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7773e-05 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724499+0000", 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1844999999999997e-05 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718861+0000", 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.468344706, 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057310310000000001, 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.268 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 73 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718861+0000", 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718861+0000", 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718861+0000", 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.99e-07 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718866+0000", 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718875+0000", 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6940000000000001e-06 2024-09-14T06:59:31.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718886+0000", 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0740999999999999e-05 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722979+0000", 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040930489999999996 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723023+0000", 2024-09-14T06:59:31.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4310000000000001e-05 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723091+0000", 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7656000000000004e-05 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724508+0000", 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001416622 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724527+0000", 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9072e-05 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724564+0000", 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7527999999999997e-05 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724592+0000", 2024-09-14T06:59:31.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7622000000000001e-05 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718898+0000", 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.468307709000001, 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061129890000000001, 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718898+0000", 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718898+0000", 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718898+0000", 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4499999999999997e-07 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718906+0000", 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5449999999999998e-06 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718915+0000", 2024-09-14T06:59:31.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9060000000000008e-06 2024-09-14T06:59:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718921+0000", 2024-09-14T06:59:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7859999999999999e-06 2024-09-14T06:59:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723337+0000", 2024-09-14T06:59:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044150600000000002 2024-09-14T06:59:31.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723383+0000", 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6288000000000001e-05 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723471+0000", 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7967000000000007e-05 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724604+0000", 2024-09-14T06:59:31.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001133656 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724961+0000", 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000356621 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724990+0000", 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8520000000000001e-05 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725011+0000", 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1095e-05 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718932+0000", 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.468273654000001, 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006342421, 2024-09-14T06:59:31.283 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718932+0000", 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718932+0000", 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718932+0000", 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719003+0000", 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999995e-05 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719012+0000", 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5960000000000004e-06 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719018+0000", 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723736+0000", 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047176609999999997 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723782+0000", 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6320999999999999e-05 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723866+0000", 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4053000000000003e-05 2024-09-14T06:59:31.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724940+0000", 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010740649999999999 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725226+0000", 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028630000000000002 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725253+0000", 2024-09-14T06:59:31.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6849999999999999e-05 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725274+0000", 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0865000000000001e-05 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.719029+0000", 2024-09-14T06:59:31.289 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.468176319000001, 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065583459999999996, 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 76 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719029+0000", 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719029+0000", 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719030+0000", 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999997e-07 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719037+0000", 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3549999999999999e-06 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719044+0000", 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4509999999999997e-06 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719049+0000", 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:59:31.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724617+0000", 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0055679689999999999 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724658+0000", 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1028000000000001e-05 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724726+0000", 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8717e-05 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725288+0000", 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00056203299999999998 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725541+0000", 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025287999999999998 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725569+0000", 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7246999999999998e-05 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725587+0000", 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8868e-05 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T06:59:31.295 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:15.023225+0000", 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.163980303999999, 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066263060000000002, 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 92 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023225+0000", 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023225+0000", 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023225+0000", 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023228+0000", 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0930000000000001e-06 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023231+0000", 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023234+0000", 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5720000000000001e-06 2024-09-14T06:59:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025332+0000", 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002098028 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025371+0000", 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9295000000000002e-05 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025436+0000", 2024-09-14T06:59:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4725000000000001e-05 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026698+0000", 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001262752 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.029794+0000", 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030956109999999998 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.029823+0000", 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9064000000000002e-05 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.029851+0000", 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8274999999999999e-05 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T06:59:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:15.023238+0000", 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.163967649, 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077733899999999998, 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 93 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023238+0000", 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023238+0000", 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023238+0000", 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.28e-07 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023274+0000", 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5649999999999999e-05 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023278+0000", 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0060000000000003e-06 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023280+0000", 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7640000000000001e-06 2024-09-14T06:59:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025670+0000", 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023893040000000001 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025711+0000", 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1427999999999997e-05 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025781+0000", 2024-09-14T06:59:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9717999999999999e-05 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026707+0000", 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00092589699999999996 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.030967+0000", 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042597709999999999 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.030992+0000", 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5041999999999998e-05 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.031011+0000", 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9582e-05 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:15.023288+0000", 2024-09-14T06:59:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.163917305999998, 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010753171000000001, 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 94 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023288+0000", 2024-09-14T06:59:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023288+0000", 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023288+0000", 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023290+0000", 2024-09-14T06:59:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0439999999999998e-06 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023299+0000", 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8289999999999997e-06 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023302+0000", 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.005e-06 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026351+0000", 2024-09-14T06:59:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030484119999999999 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026402+0000", 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1022e-05 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026471+0000", 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9675e-05 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026976+0000", 2024-09-14T06:59:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050500999999999996 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.033999+0000", 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070224730000000004 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.034024+0000", 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729e-05 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.034041+0000", 2024-09-14T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7710000000000002e-05 2024-09-14T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.313 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:31.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:59:31.315 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:31.313+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:31.415 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:31.415+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:31.463 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:31.464 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:31.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:59:31.516 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:31.515+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:31.580 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:31.580 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:31.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:59:31.618 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:31.617+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:31.718 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:31.718+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:31.737 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4272.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T06:59:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:20.121031+0000", 2024-09-14T06:59:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.616098682000001, 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089234399999999999, 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121031+0000", 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121031+0000", 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121031+0000", 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4499999999999998e-07 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121038+0000", 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8410000000000002e-06 2024-09-14T06:59:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121052+0000", 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.382e-05 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121060+0000", 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121121+0000", 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0642e-05 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121144+0000", 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3198999999999999e-05 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121864+0000", 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072071199999999998 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121923+0000", 2024-09-14T06:59:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9015000000000003e-05 2024-09-14T06:59:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:31.742 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:31.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:59:31.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:59:31.831 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:31.830+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:31.873 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:31.873 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:31.891 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:31.891 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:31.899 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:31.899 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:31.899 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:31.899 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:31.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:59:31.909 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:31.909 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:31.921 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:31.920+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:31.925 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:31.925 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:31.942 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:31.942 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:31.960 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:31.977 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:31.977 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:31.995 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:32.033 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:32.033 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:32.034 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:32.033+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:32.042 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:59:32.126 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:32.125+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:32.189 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:32.189 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:32.198 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:59:32.242 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:32.242+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:32.311 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:59:32.312 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:59:32.312 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T06:59:32.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:59:32.313 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:32.313 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:32.322 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:59:32.364 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:32.363+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:32.424 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:32.423+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:32.467 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:32.467 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:32.476 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:59:32.525 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:32.524+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:1 writesame 0x409849a thru 0x40a7d81 data_size 0xb4(0xf8e8 bytes) 2024-09-14T06:59:32.547 INFO:teuthology.orchestra.run.smithi007.stdout:2 write 0xda6f08 thru 0xdaafbc (0x40b5 bytes) 2024-09-14T06:59:32.550 INFO:teuthology.orchestra.run.smithi007.stdout:3 writesame 0x22aa461 thru 0x22b18bc data_size 0x16(0x745c bytes) 2024-09-14T06:59:32.561 INFO:teuthology.orchestra.run.smithi007.stdout:5 write 0x5a709bc thru 0x5a762d5 (0x591a bytes) 2024-09-14T06:59:32.564 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0x95df797 2024-09-14T06:59:32.569 INFO:teuthology.orchestra.run.smithi007.stdout:6 trunc from 0x5a762d6 to 0x95df797 2024-09-14T06:59:32.575 INFO:teuthology.orchestra.run.smithi007.stdout:7 read 0x4d4bbaf thru 0x4d56e79 (0xb2cb bytes) 2024-09-14T06:59:32.575 INFO:teuthology.orchestra.run.smithi007.stdout:8 read 0x27f9715 thru 0x280833d (0xec29 bytes) 2024-09-14T06:59:32.575 INFO:teuthology.orchestra.run.smithi007.stdout:9 read 0x36af8d8 thru 0x36b8da4 (0x94cd bytes) 2024-09-14T06:59:32.578 INFO:teuthology.orchestra.run.smithi007.stdout:10 trunc from 0x95df797 to 0x699ab06 2024-09-14T06:59:32.581 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:32.581 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:32.583 INFO:teuthology.orchestra.run.smithi007.stdout:13 punch from 0x661aac6 to 0x661c093, (0x15cd bytes) 2024-09-14T06:59:32.584 INFO:teuthology.orchestra.run.smithi007.stdout:15 read 0x3374e8f thru 0x337c140 (0x72b2 bytes) 2024-09-14T06:59:32.584 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.584 INFO:teuthology.orchestra.run.smithi007.stdout:17 write 0x234b7c0 thru 0x2358a94 (0xd2d5 bytes) 2024-09-14T06:59:32.585 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.585 INFO:teuthology.orchestra.run.smithi007.stdout:20 write 0x5e6508c thru 0x5e72661 (0xd5d6 bytes) 2024-09-14T06:59:32.589 INFO:teuthology.orchestra.run.smithi007.stdout:21 read 0x12fc378 thru 0x12fe104 (0x1d8d bytes) 2024-09-14T06:59:32.589 INFO:teuthology.orchestra.run.smithi007.stdout:22 read 0x5e7bcbc thru 0x5e810fa (0x543f bytes) 2024-09-14T06:59:32.591 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:59:32.594 INFO:teuthology.orchestra.run.smithi007.stdout:24 writesame 0x6dabaf7 thru 0x6db2ea3 data_size 0x1(0x73ad bytes) 2024-09-14T06:59:32.608 INFO:teuthology.orchestra.run.smithi007.stdout:26 write 0xe0a2f31 thru 0xe0a38c8 (0x998 bytes) 2024-09-14T06:59:32.611 INFO:teuthology.orchestra.run.smithi007.stdout:27 writesame 0x6b4d20e thru 0x6b5ad26 data_size 0x1(0xdb19 bytes) 2024-09-14T06:59:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:32.630 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 27, 2024-09-14T06:59:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6", 2024-09-14T06:59:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-14T06:58:59.479055+0000", 2024-09-14T06:59:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-14T06:59:30.798342+0000", 2024-09-14T06:59:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-14T06:59:11.718934+0000", 2024-09-14T06:59:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-14T06:59:02.486396+0000", 2024-09-14T06:59:32.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5806080, 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 6, 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:59:32.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:11.765419+0000", 2024-09-14T06:59:32.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:59:32.633 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T06:59:32.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "18", 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T06:59:32.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T06:59:32.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T06:59:32.637 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 8, 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 8, 2024-09-14T06:59:32.638 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.25, 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:18.343475+0000", 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T06:59:32.639 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-14T06:59:32.640 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "22", 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:59:32.641 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 22, 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T06:59:32.642 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:59:32.643 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T06:59:32.644 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 4.9800000190734863, 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 4.9800000190734863, 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:27.257805+0000", 2024-09-14T06:59:32.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-14T06:59:32.646 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "27", 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T06:59:32.647 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 27, 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T06:59:32.648 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T06:59:32.649 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T06:59:32.650 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.75, 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.75, 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.75, 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.75, 2024-09-14T06:59:32.651 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c036038d-cffb-4584-8851-69604a31ee40", 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:32.652 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:32.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-14T06:59:32.654 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-14T06:59:32.655 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/2081065366", 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/2081065366", 2024-09-14T06:59:32.657 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/2081065366", 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/2081065366", 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "1c80080f-932a-4957-b64b-f30c4bdb9340", 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:32.658 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:32.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6806/1688050909", 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6810/1688050909", 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/1688050909", 2024-09-14T06:59:32.663 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/1688050909", 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f3544d11-8a0f-498c-a03c-5d34134c9773", 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:32.664 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/1872259061", 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/1872259061", 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/1872259061", 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/1872259061", 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:32.669 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d2261455-c91b-414b-925f-388f0e094dbe", 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:32.670 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:32.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:32.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:32.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:32.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/1193301030", 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/1193301030", 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6811/1193301030", 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6807/1193301030", 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T06:59:32.675 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4088d23b-0d06-4415-9eba-6fb64355746c", 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6810", 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6811", 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6812", 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6813", 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6816", 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6817", 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6814", 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6815", 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6811/2283562461", 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6813/2283562461", 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6817/2283562461", 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6815/2283562461", 2024-09-14T06:59:32.680 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "5e27cb53-ecce-4061-bc5a-be599bf18cc4", 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T06:59:32.681 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6818", 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6819", 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.682 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6820", 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6821", 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.683 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6824", 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6825", 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6822", 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6823", 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6819/2212769626", 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6821/2212769626", 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6825/2212769626", 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6823/2212769626", 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ceb9a1f1-f54c-48d5-91d8-2287cac96ceb", 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:32.686 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6826", 2024-09-14T06:59:32.687 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6827", 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6828", 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6829", 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6832", 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6833", 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6830", 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6831", 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6827/3197231389", 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6829/3197231389", 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6833/3197231389", 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6831/3197231389", 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T06:59:32.692 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85c59cd6-6e2b-4cc0-852d-d1f075a8826d", 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6800", 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6801", 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:32.694 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6802", 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6803", 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6806", 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6807", 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.696 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6804", 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6805", 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6801/2162534312", 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6803/2162534312", 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6807/2162534312", 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6805/2162534312", 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-14T06:59:32.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.064941+0000", 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:32.699 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.780598+0000", 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.127700+0000", 2024-09-14T06:59:32.700 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.730995+0000", 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T06:59:32.701 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.864423+0000", 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:32.702 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.329361+0000", 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:10.107581+0000", 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.300043+0000", 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-14T06:59:32.704 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.705 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 2, 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:32.706 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 2, 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:59:32.707 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-14T06:59:32.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:32.708 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:32.708 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:32.634+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:32.709 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.709 INFO:teuthology.orchestra.run.smithi007.stdout:31 write 0x9ab3db8 thru 0x9ab883c (0x4a85 bytes) 2024-09-14T06:59:32.710 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:32.710 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:32.711 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:59:32.712 INFO:tasks.thrashosds.thrasher:j is {'epoch': 27, 'fsid': '32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6', 'created': '2024-09-14T06:58:59.479055+0000', 'modified': '2024-09-14T06:59:30.798342+0000', 'last_up_change': '2024-09-14T06:59:11.718934+0000', 'last_in_change': '2024-09-14T06:59:02.486396+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:59:11.765419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:59:18.343475+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:59:27.257805+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 27, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c036038d-cffb-4584-8851-69604a31ee40', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 2081065366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 2081065366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 2081065366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 2081065366}]}, 'public_addr': '172.21.15.7:6817/2081065366', 'cluster_addr': '172.21.15.7:6819/2081065366', 'heartbeat_back_addr': '172.21.15.7:6823/2081065366', 'heartbeat_front_addr': '172.21.15.7:6821/2081065366', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1c80080f-932a-4957-b64b-f30c4bdb9340', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6806', 'nonce': 1688050909}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6810', 'nonce': 1688050909}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 1688050909}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 1688050909}]}, 'public_addr': '172.21.15.7:6806/1688050909', 'cluster_addr': '172.21.15.7:6810/1688050909', 'heartbeat_back_addr': '172.21.15.7:6815/1688050909', 'heartbeat_front_addr': '172.21.15.7:6813/1688050909', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f3544d11-8a0f-498c-a03c-5d34134c9773', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 1872259061}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 1872259061}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 1872259061}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 1872259061}]}, 'public_addr': '172.21.15.7:6825/1872259061', 'cluster_addr': '172.21.15.7:6827/1872259061', 'heartbeat_back_addr': '172.21.15.7:6831/1872259061', 'heartbeat_front_addr': '172.21.15.7:6829/1872259061', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd2261455-c91b-414b-925f-388f0e094dbe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 1193301030}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 1193301030}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6809', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 1193301030}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6805', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 1193301030}]}, 'public_addr': '172.21.15.7:6801/1193301030', 'cluster_addr': '172.21.15.7:6803/1193301030', 'heartbeat_back_addr': '172.21.15.7:6811/1193301030', 'heartbeat_front_addr': '172.21.15.7:6807/1193301030', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4088d23b-0d06-4415-9eba-6fb64355746c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6810', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6811', 'nonce': 2283562461}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6812', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6813', 'nonce': 2283562461}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6816', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6817', 'nonce': 2283562461}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6814', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6815', 'nonce': 2283562461}]}, 'public_addr': '172.21.15.189:6811/2283562461', 'cluster_addr': '172.21.15.189:6813/2283562461', 'heartbeat_back_addr': '172.21.15.189:6817/2283562461', 'heartbeat_front_addr': '172.21.15.189:6815/2283562461', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5e27cb53-ecce-4061-bc5a-be599bf18cc4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6818', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6819', 'nonce': 2212769626}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6820', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6821', 'nonce': 2212769626}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6824', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6825', 'nonce': 2212769626}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6822', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6823', 'nonce': 2212769626}]}, 'public_addr': '172.21.15.189:6819/2212769626', 'cluster_addr': '172.21.15.189:6821/2212769626', 'heartbeat_back_addr': '172.21.15.189:6825/2212769626', 'heartbeat_front_addr': '172.21.15.189:6823/2212769626', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceb9a1f1-f54c-48d5-91d8-2287cac96ceb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6826', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6827', 'nonce': 3197231389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6828', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6829', 'nonce': 3197231389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 3197231389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6830', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6831', 'nonce': 3197231389}]}, 'public_addr': '172.21.15.189:6827/3197231389', 'cluster_addr': '172.21.15.189:6829/3197231389', 'heartbeat_back_addr': '172.21.15.189:6833/3197231389', 'heartbeat_front_addr': '172.21.15.189:6831/3197231389', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '85c59cd6-6e2b-4cc0-852d-d1f075a8826d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6800', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6801', 'nonce': 2162534312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6802', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6803', 'nonce': 2162534312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6806', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6807', 'nonce': 2162534312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6804', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6805', 'nonce': 2162534312}]}, 'public_addr': '172.21.15.189:6801/2162534312', 'cluster_addr': '172.21.15.189:6803/2162534312', 'heartbeat_back_addr': '172.21.15.189:6807/2162534312', 'heartbeat_front_addr': '172.21.15.189:6805/2162534312', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.064941+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.780598+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.127700+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.730995+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.864423+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.329361+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:10.107581+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.300043+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:59:32.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:59:32.714 INFO:teuthology.orchestra.run.smithi007.stdout:32 write 0x3b19266 thru 0x3b28e52 (0xfbed bytes) 2024-09-14T06:59:32.714 INFO:teuthology.orchestra.run.smithi007.stdout:33 write 0xb122c69 thru 0xb13249c (0xf834 bytes) 2024-09-14T06:59:32.714 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:36 write 0x4b19f42 thru 0x4b1de36 (0x3ef5 bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:40 punch from 0x1e853a0 to 0x1e8ae06, (0x5a66 bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:41 read 0xfe9952 thru 0xfe9d37 (0x3e6 bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:42 read 0xcc267d5 thru 0xcc2992a (0x3156 bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:44 read 0xb5c18e8 thru 0xb5cc829 (0xaf42 bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:45 punch from 0x9adcf74 to 0x9ae2899, (0x5925 bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:46 read 0x1372191 thru 0x1375013 (0x2e83 bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:47 write 0x3ba7aae thru 0x3ba938c (0x18df bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:49 read 0x24f38d3 thru 0x24f9138 (0x5866 bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:50 read 0x6e31b1a thru 0x6e32ae9 (0xfd0 bytes) 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.715 INFO:teuthology.orchestra.run.smithi007.stdout:53 write 0xa9e542a thru 0xa9f2c0a (0xd7e1 bytes) 2024-09-14T06:59:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:56 write 0xba9a1e0 thru 0xbaa9eac (0xfccd bytes) 2024-09-14T06:59:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:57 writesame 0xd6101e9 thru 0xd6159f0 data_size 0x4(0x5808 bytes) 2024-09-14T06:59:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:59 punch from 0x7cad8a to 0x7d3239, (0x84af bytes) 2024-09-14T06:59:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:60 read 0x4e6bbdf thru 0x4e78892 (0xccb4 bytes) 2024-09-14T06:59:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:61 trunc from 0xe0a38c9 to 0x39721d 2024-09-14T06:59:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:62 read 0x280560 thru 0x2842a8 (0x3d49 bytes) 2024-09-14T06:59:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:63 writesame 0x8da4488 thru 0x8da6933 data_size 0x2(0x24ac bytes) 2024-09-14T06:59:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:64 write 0x87c441f thru 0x87caa12 (0x65f4 bytes) 2024-09-14T06:59:32.722 INFO:teuthology.orchestra.run.smithi007.stdout:65 writesame 0xaa33b1e thru 0xaa33e00 data_size 0x1(0x2e3 bytes) 2024-09-14T06:59:32.722 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0xcf5f5be 2024-09-14T06:59:32.725 INFO:teuthology.orchestra.run.smithi007.stdout:67 trunc from 0xaa33e01 to 0xcf5f5be 2024-09-14T06:59:32.726 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:32.725+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:32.734 INFO:teuthology.orchestra.run.smithi007.stdout:68 trunc from 0xcf5f5be to 0x9861f04 2024-09-14T06:59:32.740 INFO:teuthology.orchestra.run.smithi007.stdout:69 read 0x9753732 thru 0x97557ba (0x2089 bytes) 2024-09-14T06:59:32.740 INFO:teuthology.orchestra.run.smithi007.stdout:70 writesame 0x37c966b thru 0x37ce496 data_size 0x2(0x4e2c bytes) 2024-09-14T06:59:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.747 INFO:teuthology.orchestra.run.smithi007.stdout:73 trunc from 0x9861f04 to 0x93a71f8 2024-09-14T06:59:32.753 INFO:teuthology.orchestra.run.smithi007.stdout:75 punch from 0x54227c6 to 0x54245a6, (0x1de0 bytes) 2024-09-14T06:59:32.753 INFO:teuthology.orchestra.run.smithi007.stdout:76 write 0x8fd0eac thru 0x8fda5ea (0x973f bytes) 2024-09-14T06:59:32.757 INFO:teuthology.orchestra.run.smithi007.stdout:78 write 0xc00254 thru 0xc078c1 (0x766e bytes) 2024-09-14T06:59:32.760 INFO:teuthology.orchestra.run.smithi007.stdout:truncating to largest ever: 0xedf5467 2024-09-14T06:59:32.767 INFO:teuthology.orchestra.run.smithi007.stdout:79 trunc from 0x93a71f8 to 0xedf5467 2024-09-14T06:59:32.773 INFO:teuthology.orchestra.run.smithi007.stdout:80 read 0x2efa5bd thru 0x2f0704b (0xca8f bytes) 2024-09-14T06:59:32.776 INFO:teuthology.orchestra.run.smithi007.stdout:82 trunc from 0xedf5467 to 0xc15f4ab 2024-09-14T06:59:32.781 INFO:teuthology.orchestra.run.smithi007.stdout:83 read 0x15b9e97 thru 0x15c54f8 (0xb662 bytes) 2024-09-14T06:59:32.781 INFO:teuthology.orchestra.run.smithi007.stdout:84 punch from 0xbd07881 to 0xbd12f5f, (0xb6de bytes) 2024-09-14T06:59:32.783 INFO:teuthology.orchestra.run.smithi007.stdout:85 trunc from 0xc15f4ab to 0xd78cb0a 2024-09-14T06:59:32.790 INFO:teuthology.orchestra.run.smithi007.stdout:87 read 0x60f084 thru 0x61b7c6 (0xc743 bytes) 2024-09-14T06:59:32.790 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.790 INFO:teuthology.orchestra.run.smithi007.stdout:89 punch from 0xcd9f406 to 0xcda3432, (0x402c bytes) 2024-09-14T06:59:32.790 INFO:teuthology.orchestra.run.smithi007.stdout:91 punch from 0x91fbf96 to 0x92080c5, (0xc12f bytes) 2024-09-14T06:59:32.790 INFO:teuthology.orchestra.run.smithi007.stdout:92 punch from 0x3bef943 to 0x3bf1131, (0x17ee bytes) 2024-09-14T06:59:32.790 INFO:teuthology.orchestra.run.smithi007.stdout:93 write 0xb4a7a9c thru 0xb4b2bb2 (0xb117 bytes) 2024-09-14T06:59:32.793 INFO:teuthology.orchestra.run.smithi007.stdout:95 read 0x95e3b12 thru 0x95e9b26 (0x6015 bytes) 2024-09-14T06:59:32.793 INFO:teuthology.orchestra.run.smithi007.stdout:97 read 0x833b01f thru 0x833dcda (0x2cbc bytes) 2024-09-14T06:59:32.793 INFO:teuthology.orchestra.run.smithi007.stdout:98 read 0x47df4f7 thru 0x47ed599 (0xe0a3 bytes) 2024-09-14T06:59:32.793 INFO:teuthology.orchestra.run.smithi007.stdout:100 write 0xaebf59c thru 0xaebfdd8 (0x83d bytes) 2024-09-14T06:59:32.796 INFO:teuthology.orchestra.run.smithi007.stdout:102 read 0xb02fe39 thru 0xb039100 (0x92c8 bytes) 2024-09-14T06:59:32.796 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.796 INFO:teuthology.orchestra.run.smithi007.stdout:105 read 0x4773509 thru 0x477c388 (0x8e80 bytes) 2024-09-14T06:59:32.796 INFO:teuthology.orchestra.run.smithi007.stdout:106 write 0x4953198 thru 0x4956a7d (0x38e6 bytes) 2024-09-14T06:59:32.799 INFO:teuthology.orchestra.run.smithi007.stdout:107 trunc from 0xd78cb0a to 0xd132fb1 2024-09-14T06:59:32.804 INFO:teuthology.orchestra.run.smithi007.stdout:109 read 0xafd2e3c thru 0xafd9a7d (0x6c42 bytes) 2024-09-14T06:59:32.805 INFO:teuthology.orchestra.run.smithi007.stdout:110 read 0x7f981d2 thru 0x7f99353 (0x1182 bytes) 2024-09-14T06:59:32.805 INFO:teuthology.orchestra.run.smithi007.stdout:111 write 0x48543e0 thru 0x48599fb (0x561c bytes) 2024-09-14T06:59:32.806 INFO:teuthology.orchestra.run.smithi007.stdout:112 punch from 0x4f3b798 to 0x4f3bfee, (0x856 bytes) 2024-09-14T06:59:32.806 INFO:teuthology.orchestra.run.smithi007.stdout:113 write 0x16eaddf thru 0x16eef20 (0x4142 bytes) 2024-09-14T06:59:32.810 INFO:teuthology.orchestra.run.smithi007.stdout:114 read 0x9fe1481 thru 0x9fec4a2 (0xb022 bytes) 2024-09-14T06:59:32.810 INFO:teuthology.orchestra.run.smithi007.stdout:115 read 0xb27a035 thru 0xb27e5a5 (0x4571 bytes) 2024-09-14T06:59:32.813 INFO:teuthology.orchestra.run.smithi007.stdout:118 trunc from 0xd132fb1 to 0xa867941 2024-09-14T06:59:32.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:59:32.823 INFO:teuthology.orchestra.run.smithi007.stdout:119 write 0x95627c9 thru 0x956bf84 (0x97bc bytes) 2024-09-14T06:59:32.826 INFO:teuthology.orchestra.run.smithi007.stdout:120 read 0x4fae25d thru 0x4fb092e (0x26d2 bytes) 2024-09-14T06:59:32.826 INFO:teuthology.orchestra.run.smithi007.stdout:121 writesame 0x6012620 thru 0x601b72d data_size 0x6(0x910e bytes) 2024-09-14T06:59:32.826 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:32.826+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:32.844 INFO:teuthology.orchestra.run.smithi007.stdout:124 writesame 0xe988465 thru 0xe98d8ad data_size 0x1(0x5449 bytes) 2024-09-14T06:59:32.844 INFO:teuthology.orchestra.run.smithi007.stdout:125 write 0x2435814 thru 0x243a9e8 (0x51d5 bytes) 2024-09-14T06:59:32.853 INFO:teuthology.orchestra.run.smithi007.stdout:126 write 0xea38f30 thru 0xea4616e (0xd23f bytes) 2024-09-14T06:59:32.856 INFO:teuthology.orchestra.run.smithi007.stdout:128 write 0xf208ba thru 0xf30313 (0xfa5a bytes) 2024-09-14T06:59:32.857 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:32.857 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:32.858 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:32.858 INFO:teuthology.orchestra.run.smithi007.stdout:130 read 0xd28373c thru 0xd284dd1 (0x1696 bytes) 2024-09-14T06:59:32.867 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:59:32.871 INFO:teuthology.orchestra.run.smithi007.stdout:132 trunc from 0xea4616f to 0x37a92aa 2024-09-14T06:59:32.884 INFO:teuthology.orchestra.run.smithi007.stdout:133 read 0x245a2c4 thru 0x246078f (0x64cc bytes) 2024-09-14T06:59:32.884 INFO:teuthology.orchestra.run.smithi007.stdout:134 clone 1 order 25 su 1048576 sc 6 2024-09-14T06:59:32.951 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:32.950+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:33.011 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:33.011 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:33.019 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:59:33.047 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:33.048 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T06:59:33.061 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:33.060+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:33.064 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-14T06:59:31.752637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":42,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234912,"kb_used_data":2032,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739808,"statfs":{"total":773094113280,"available":772853563392,"internally_reserved":0,"allocated":2080768,"data_stored":1875148,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000819"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-14T06:59:29.800365+0000","last_change":"2024-09-14T06:59:28.802517+0000","last_active":"2024-09-14T06:59:29.800365+0000","last_peered":"2024-09-14T06:59:29.800365+0000","last_clean":"2024-09-14T06:59:29.800365+0000","last_became_active":"2024-09-14T06:59:28.801961+0000","last_became_peered":"2024-09-14T06:59:28.801961+0000","last_unstale":"2024-09-14T06:59:29.800365+0000","last_undegraded":"2024-09-14T06:59:29.800365+0000","last_fullsized":"2024-09-14T06:59:29.800365+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:26:03.470684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-14T06:59:29.800229+0000","last_change":"2024-09-14T06:59:28.802044+0000","last_active":"2024-09-14T06:59:29.800229+0000","last_peered":"2024-09-14T06:59:29.800229+0000","last_clean":"2024-09-14T06:59:29.800229+0000","last_became_active":"2024-09-14T06:59:28.801728+0000","last_became_peered":"2024-09-14T06:59:28.801728+0000","last_unstale":"2024-09-14T06:59:29.800229+0000","last_undegraded":"2024-09-14T06:59:29.800229+0000","last_fullsized":"2024-09-14T06:59:29.800229+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:47.731444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T06:59:30.801680+0000","last_change":"2024-09-14T06:59:30.801680+0000","last_active":"2024-09-14T06:59:30.801680+0000","last_peered":"2024-09-14T06:59:30.801680+0000","last_clean":"2024-09-14T06:59:30.801680+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T06:59:30.801680+0000","last_undegraded":"2024-09-14T06:59:30.801680+0000","last_fullsized":"2024-09-14T06:59:30.801680+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:20:13.510181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060885900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T06:59:30.801946+0000","last_change":"2024-09-14T06:59:30.801946+0000","last_active":"2024-09-14T06:59:30.801946+0000","last_peered":"2024-09-14T06:59:30.801946+0000","last_clean":"2024-09-14T06:59:30.801946+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T06:59:30.801946+0000","last_undegraded":"2024-09-14T06:59:30.801946+0000","last_fullsized":"2024-09-14T06:59:30.801946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:02:03.974374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081998900000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-14T06:59:29.800234+0000","last_change":"2024-09-14T06:59:28.804077+0000","last_active":"2024-09-14T06:59:29.800234+0000","last_peered":"2024-09-14T06:59:29.800234+0000","last_clean":"2024-09-14T06:59:29.800234+0000","last_became_active":"2024-09-14T06:59:28.803823+0000","last_became_peered":"2024-09-14T06:59:28.803823+0000","last_unstale":"2024-09-14T06:59:29.800234+0000","last_undegraded":"2024-09-14T06:59:29.800234+0000","last_fullsized":"2024-09-14T06:59:29.800234+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:31.962450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-14T06:59:29.800370+0000","last_change":"2024-09-14T06:59:28.801582+0000","last_active":"2024-09-14T06:59:29.800370+0000","last_peered":"2024-09-14T06:59:29.800370+0000","last_clean":"2024-09-14T06:59:29.800370+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T06:59:29.800370+0000","last_undegraded":"2024-09-14T06:59:29.800370+0000","last_fullsized":"2024-09-14T06:59:29.800370+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:15:44.314062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T06:59:24.273697+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T06:59:24.273697+0000","last_peered":"2024-09-14T06:59:24.273697+0000","last_clean":"2024-09-14T06:59:24.273697+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T06:59:24.273697+0000","last_undegraded":"2024-09-14T06:59:24.273697+0000","last_fullsized":"2024-09-14T06:59:24.273697+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T06:59:30.801713+0000","last_change":"2024-09-14T06:59:30.801713+0000","last_active":"2024-09-14T06:59:30.801713+0000","last_peered":"2024-09-14T06:59:30.801713+0000","last_clean":"2024-09-14T06:59:30.801713+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T06:59:30.801713+0000","last_undegraded":"2024-09-14T06:59:30.801713+0000","last_fullsized":"2024-09-14T06:59:30.801713+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:15:31.422623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059990300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T06:59:30.801065+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T06:59:30.801065+0000","last_peered":"2024-09-14T06:59:30.801065+0000","last_clean":"2024-09-14T06:59:30.801065+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T06:59:30.801065+0000","last_undegraded":"2024-09-14T06:59:30.801065+0000","last_fullsized":"2024-09-14T06:59:30.801065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T06:59:30.801930+0000","last_change":"2024-09-14T06:59:30.801930+0000","last_active":"2024-09-14T06:59:30.801930+0000","last_peered":"2024-09-14T06:59:30.801930+0000","last_clean":"2024-09-14T06:59:30.801930+0000","last_became_active":"2024-09-14T06:59:28.803870+0000","last_became_peered":"2024-09-14T06:59:28.803870+0000","last_unstale":"2024-09-14T06:59:30.801930+0000","last_undegraded":"2024-09-14T06:59:30.801930+0000","last_fullsized":"2024-09-14T06:59:30.801930+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:32:08.478895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060862000000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T06:59:24.273502+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T06:59:24.273502+0000","last_peered":"2024-09-14T06:59:24.273502+0000","last_clean":"2024-09-14T06:59:24.273502+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T06:59:24.273502+0000","last_undegraded":"2024-09-14T06:59:24.273502+0000","last_fullsized":"2024-09-14T06:59:24.273502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-14T06:59:29.800107+0000","last_change":"2024-09-14T06:59:21.902140+0000","last_active":"2024-09-14T06:59:29.800107+0000","last_peered":"2024-09-14T06:59:29.800107+0000","last_clean":"2024-09-14T06:59:29.800107+0000","last_became_active":"2024-09-14T06:59:20.121600+0000","last_became_peered":"2024-09-14T06:59:20.121600+0000","last_unstale":"2024-09-14T06:59:29.800107+0000","last_undegraded":"2024-09-14T06:59:29.800107+0000","last_fullsized":"2024-09-14T06:59:29.800107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:29:52.276761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T06:59:24.273491+0000","last_change":"2024-09-14T06:59:21.767686+0000","last_active":"2024-09-14T06:59:24.273491+0000","last_peered":"2024-09-14T06:59:24.273491+0000","last_clean":"2024-09-14T06:59:24.273491+0000","last_became_active":"2024-09-14T06:59:20.121991+0000","last_became_peered":"2024-09-14T06:59:20.121991+0000","last_unstale":"2024-09-14T06:59:24.273491+0000","last_undegraded":"2024-09-14T06:59:24.273491+0000","last_fullsized":"2024-09-14T06:59:24.273491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:28:44.196704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-14T06:59:29.800132+0000","last_change":"2024-09-14T06:59:28.808626+0000","last_active":"2024-09-14T06:59:29.800132+0000","last_peered":"2024-09-14T06:59:29.800132+0000","last_clean":"2024-09-14T06:59:29.800132+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T06:59:29.800132+0000","last_undegraded":"2024-09-14T06:59:29.800132+0000","last_fullsized":"2024-09-14T06:59:29.800132+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:42.775788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-14T06:59:28.792556+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T06:59:28.792556+0000","last_peered":"2024-09-14T06:59:28.792556+0000","last_clean":"2024-09-14T06:59:28.792556+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T06:59:28.792556+0000","last_undegraded":"2024-09-14T06:59:28.792556+0000","last_fullsized":"2024-09-14T06:59:28.792556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T06:59:24.273745+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T06:59:24.273745+0000","last_peered":"2024-09-14T06:59:24.273745+0000","last_clean":"2024-09-14T06:59:24.273745+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T06:59:24.273745+0000","last_undegraded":"2024-09-14T06:59:24.273745+0000","last_fullsized":"2024-09-14T06:59:24.273745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T06:59:30.801433+0000","last_change":"2024-09-14T06:59:30.801433+0000","last_active":"2024-09-14T06:59:30.801433+0000","last_peered":"2024-09-14T06:59:30.801433+0000","last_clean":"2024-09-14T06:59:30.801433+0000","last_became_active":"2024-09-14T06:59:28.801659+0000","last_became_peered":"2024-09-14T06:59:28.801659+0000","last_unstale":"2024-09-14T06:59:30.801433+0000","last_undegraded":"2024-09-14T06:59:30.801433+0000","last_fullsized":"2024-09-14T06:59:30.801433+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:22.201490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037135599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T06:59:24.273508+0000","last_change":"2024-09-14T06:59:21.767446+0000","last_active":"2024-09-14T06:59:24.273508+0000","last_peered":"2024-09-14T06:59:24.273508+0000","last_clean":"2024-09-14T06:59:24.273508+0000","last_became_active":"2024-09-14T06:59:20.120193+0000","last_became_peered":"2024-09-14T06:59:20.120193+0000","last_unstale":"2024-09-14T06:59:24.273508+0000","last_undegraded":"2024-09-14T06:59:24.273508+0000","last_fullsized":"2024-09-14T06:59:24.273508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:18.122142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":112,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-14T06:59:24.273860+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T06:59:24.273860+0000","last_peered":"2024-09-14T06:59:24.273860+0000","last_clean":"2024-09-14T06:59:24.273860+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T06:59:24.273860+0000","last_undegraded":"2024-09-14T06:59:24.273860+0000","last_fullsized":"2024-09-14T06:59:24.273860+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-14T06:59:30.801426+0000","last_change":"2024-09-14T06:59:30.801426+0000","last_active":"2024-09-14T06:59:30.801426+0000","last_peered":"2024-09-14T06:59:30.801426+0000","last_clean":"2024-09-14T06:59:30.801426+0000","last_became_active":"2024-09-14T06:59:28.808414+0000","last_became_peered":"2024-09-14T06:59:28.808414+0000","last_unstale":"2024-09-14T06:59:30.801426+0000","last_undegraded":"2024-09-14T06:59:30.801426+0000","last_fullsized":"2024-09-14T06:59:30.801426+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:54:27.928379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038565499999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-14T06:59:28.808198+0000","last_change":"2024-09-14T06:59:28.802982+0000","last_active":"2024-09-14T06:59:28.808198+0000","last_peered":"2024-09-14T06:59:28.808198+0000","last_clean":"2024-09-14T06:59:28.808198+0000","last_became_active":"2024-09-14T06:59:28.802658+0000","last_became_peered":"2024-09-14T06:59:28.802658+0000","last_unstale":"2024-09-14T06:59:28.808198+0000","last_undegraded":"2024-09-14T06:59:28.808198+0000","last_fullsized":"2024-09-14T06:59:28.808198+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:30:47.036793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:59:27.785355+0000","last_change":"2024-09-14T06:59:27.785355+0000","last_active":"2024-09-14T06:59:27.785355+0000","last_peered":"2024-09-14T06:59:27.785355+0000","last_clean":"2024-09-14T06:59:27.785355+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:59:27.785355+0000","last_undegraded":"2024-09-14T06:59:27.785355+0000","last_fullsized":"2024-09-14T06:59:27.785355+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":81435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":237568,"data_stored":81435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":81416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31476,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340364,"statfs":{"total":96636764160,"available":96604532736,"internally_reserved":0,"allocated":290816,"data_stored":671784,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":233472,"data_stored":81416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":331776,"data_stored":690212,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":93725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":258048,"data_stored":93725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":8,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:59:33.066 INFO:tasks.thrashosds.thrasher:Setting 3.18 to [0, 5] 2024-09-14T06:59:33.066 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.18', '0', '5'] 2024-09-14T06:59:33.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.18 0 5 2024-09-14T06:59:33.149 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:33.148+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:33.163 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:33.163 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:33.172 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:59:33.231 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:33.230+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:33.276 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:33.276 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:33.284 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:59:33.370 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:33.369+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:33.429 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:33.429 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:33.432 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:33.431+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:33.437 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:59:33.533 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:33.533+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:33.581 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:33.582 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:33.590 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:59:33.635 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:33.634+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:33.733 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:33.733 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:33.735 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:33.735+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:33.742 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:59:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T06:59:33.815 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.18 pg_upmap mapping to [0,5] 2024-09-14T06:59:33.846 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:33.847 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:33.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:59:33.875 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:33.874+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:33.938 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:33.937+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:33.986 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:33.986 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:33.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:59:34.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:59:34.040 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:34.039+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:34.104 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:34.104 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:34.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:59:34.159 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:34.158+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:34.166 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:34.184 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:34.200 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:34.201 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:34.217 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:34.226 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.226 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4431.0:10 3.1c e27/24 3:3831950c:::rbd_directory:head v 27'1, mlcod=0'0)", 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.808696+0000", 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.4175593790000001, 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0009881759999999999, 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.808696+0000", 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.808696+0000", 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.808697+0000", 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.808701+0000", 2024-09-14T06:59:34.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4139999999999996e-06 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.808714+0000", 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3219e-05 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.808725+0000", 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0784999999999999e-05 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.808806+0000", 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1367999999999998e-05 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.808825+0000", 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8909000000000001e-05 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.809640+0000", 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00081462199999999998 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.809684+0000", 2024-09-14T06:59:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4539000000000002e-05 2024-09-14T06:59:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.231 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:34.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:59:34.235 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:34.236 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:34.241 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:34.240+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:34.252 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:34.269 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:34.286 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:34.342 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:34.341+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:34.361 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:34.361 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:34.361 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:34.361 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:34.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:59:34.442 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:34.441+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:34.477 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:34.477 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:34.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:59:34.570 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:34.569+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:34.635 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:34.635 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:34.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:34.635 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.899525523000001, 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T06:59:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T06:59:34.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T06:59:34.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T06:59:34.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414215+0000", 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.219027294, 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0062819, 2024-09-14T06:59:34.643 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 46 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414215+0000", 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414215+0000", 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414215+0000", 2024-09-14T06:59:34.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1300000000000001e-07 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414219+0000", 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0160000000000002e-06 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414227+0000", 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414235+0000", 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3860000000000007e-06 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418085+0000", 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00385054 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418135+0000", 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9163999999999999e-05 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418212+0000", 2024-09-14T06:59:34.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6926000000000002e-05 2024-09-14T06:59:34.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419422+0000", 2024-09-14T06:59:34.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001210123 2024-09-14T06:59:34.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420383+0000", 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000961596 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420467+0000", 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3967000000000004e-05 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420497+0000", 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9331000000000001e-05 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414255+0000", 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.218986994000002, 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069405730000000002, 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.650 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 47 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414255+0000", 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414255+0000", 2024-09-14T06:59:34.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414256+0000", 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-07 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414347+0000", 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1472999999999996e-05 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414357+0000", 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414364+0000", 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.584e-06 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418461+0000", 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040973720000000002 2024-09-14T06:59:34.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418550+0000", 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8548999999999995e-05 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.418682+0000", 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013265600000000001 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420504+0000", 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018218150000000001 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421143+0000", 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063939300000000004 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421174+0000", 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0624999999999999e-05 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421196+0000", 2024-09-14T06:59:34.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1454000000000002e-05 2024-09-14T06:59:34.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.656 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414378+0000", 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.218864292999999, 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078204360000000001, 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 48 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414378+0000", 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414378+0000", 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414378+0000", 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414383+0000", 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6110000000000004e-06 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414390+0000", 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5000000000000002e-06 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414398+0000", 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3119999999999996e-06 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419441+0000", 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050428770000000003 2024-09-14T06:59:34.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419489+0000", 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7877999999999998e-05 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419562+0000", 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3547999999999994e-05 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420507+0000", 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094460900000000001 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422088+0000", 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015811250000000001 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422152+0000", 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4238000000000004e-05 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422198+0000", 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6091000000000003e-05 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414416+0000", 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.218826119999999, 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078677350000000007, 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 49 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.664 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414416+0000", 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414416+0000", 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414416+0000", 2024-09-14T06:59:34.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414421+0000", 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3680000000000004e-06 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414429+0000", 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9859999999999994e-06 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414435+0000", 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8089999999999998e-06 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419804+0000", 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005369497 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.419905+0000", 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100521 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420051+0000", 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014670199999999999 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422028+0000", 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019769380000000001 2024-09-14T06:59:34.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422232+0000", 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020340700000000001 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422259+0000", 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422284+0000", 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4573e-05 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414454+0000", 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.218788060000001, 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079161430000000005, 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 50 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414454+0000", 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414454+0000", 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414455+0000", 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:59:34.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414574+0000", 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119368 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414581+0000", 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8950000000000001e-06 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414589+0000", 2024-09-14T06:59:34.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2449999999999998e-06 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420538+0000", 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0059490569999999998 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420648+0000", 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011021799999999999 2024-09-14T06:59:34.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.420765+0000", 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116749 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422290+0000", 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001525059 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422314+0000", 2024-09-14T06:59:34.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3533999999999999e-05 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422344+0000", 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0334000000000001e-05 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422370+0000", 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6069000000000001e-05 2024-09-14T06:59:34.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414598+0000", 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.218643948, 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080802319999999997, 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 51 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414598+0000", 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414598+0000", 2024-09-14T06:59:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414598+0000", 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6100000000000001e-07 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414601+0000", 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414606+0000", 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1410000000000004e-06 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414609+0000", 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.982e-06 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421243+0000", 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006634175 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421321+0000", 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7647999999999999e-05 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421440+0000", 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011891 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422381+0000", 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094076700000000001 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422629+0000", 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000247972 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422658+0000", 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9152000000000001e-05 2024-09-14T06:59:34.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422678+0000", 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0517e-05 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.414640+0000", 2024-09-14T06:59:34.683 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.218601979999999, 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081134169999999995, 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 52 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414640+0000", 2024-09-14T06:59:34.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414640+0000", 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414640+0000", 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414643+0000", 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414650+0000", 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5259999999999999e-06 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.414653+0000", 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0149999999999999e-06 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421706+0000", 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070536139999999997 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421755+0000", 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8702999999999998e-05 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.421839+0000", 2024-09-14T06:59:34.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3554999999999995e-05 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422386+0000", 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00054785700000000001 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422706+0000", 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031942199999999998 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422734+0000", 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8411999999999998e-05 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.422753+0000", 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120999999999999e-05 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.689 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:54 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 131072~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.423043+0000", 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.210198738999999, 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054059049999999999, 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 54 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423043+0000", 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423043+0000", 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423044+0000", 2024-09-14T06:59:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:59:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423105+0000", 2024-09-14T06:59:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1645999999999999e-05 2024-09-14T06:59:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423112+0000", 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0839999999999997e-06 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423116+0000", 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9840000000000003e-06 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423529+0000", 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041364 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423609+0000", 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0133e-05 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423745+0000", 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013610599999999999 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424644+0000", 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089827699999999995 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428393+0000", 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037495390000000001 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428427+0000", 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3852e-05 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428449+0000", 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2067000000000001e-05 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:55 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 196608~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.423126+0000", 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.210115928, 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054169220000000002, 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 55 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423126+0000", 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423126+0000", 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423126+0000", 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2599999999999998e-07 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423187+0000", 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0185000000000001e-05 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423192+0000", 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4979999999999997e-06 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423196+0000", 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1649999999999999e-06 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424071+0000", 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087498000000000003 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424118+0000", 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6912000000000001e-05 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424228+0000", 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110087 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.425125+0000", 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089658999999999995 2024-09-14T06:59:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428484+0000", 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00335907 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428517+0000", 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3311e-05 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.428543+0000", 2024-09-14T06:59:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5698e-05 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:56 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.423204+0000", 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.210037818, 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008228282, 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 56 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423204+0000", 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423204+0000", 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423205+0000", 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-07 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423243+0000", 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8618000000000002e-05 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423248+0000", 2024-09-14T06:59:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.423256+0000", 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424676+0000", 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001420227 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424741+0000", 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4943000000000005e-05 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.424820+0000", 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8654999999999998e-05 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.425403+0000", 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058315400000000001 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.431305+0000", 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0059021509999999996 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.431377+0000", 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2078999999999999e-05 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.431432+0000", 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5275999999999997e-05 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718658+0000", 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.914583682, 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005725178, 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 71 2024-09-14T06:59:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718658+0000", 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718658+0000", 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718659+0000", 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718663+0000", 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.245e-06 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718689+0000", 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5982e-05 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718696+0000", 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0219999999999999e-06 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.721646+0000", 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029494539999999998 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.721690+0000", 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3968000000000003e-05 2024-09-14T06:59:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.721758+0000", 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8134999999999997e-05 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722880+0000", 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011223540000000001 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724334+0000", 2024-09-14T06:59:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00145371 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724362+0000", 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8158e-05 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724383+0000", 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1574999999999999e-05 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718712+0000", 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.914529642000002, 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057869210000000004, 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 72 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718712+0000", 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718712+0000", 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718713+0000", 2024-09-14T06:59:34.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718818+0000", 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105278 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718835+0000", 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6779999999999999e-05 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718844+0000", 2024-09-14T06:59:34.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4989999999999994e-06 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722033+0000", 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031890360000000001 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722077+0000", 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4919999999999997e-05 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722153+0000", 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5764000000000002e-05 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723304+0000", 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011504379999999999 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724430+0000", 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011259760000000001 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724457+0000", 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7773e-05 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724499+0000", 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1844999999999997e-05 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.719 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718861+0000", 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.914381294000002, 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057310310000000001, 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 73 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718861+0000", 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718861+0000", 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718861+0000", 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.99e-07 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718866+0000", 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718875+0000", 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6940000000000001e-06 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718886+0000", 2024-09-14T06:59:34.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0740999999999999e-05 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.722979+0000", 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0040930489999999996 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723023+0000", 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4310000000000001e-05 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723091+0000", 2024-09-14T06:59:34.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7656000000000004e-05 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724508+0000", 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001416622 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724527+0000", 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9072e-05 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724564+0000", 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7527999999999997e-05 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724592+0000", 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7622000000000001e-05 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718898+0000", 2024-09-14T06:59:34.725 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.914344297, 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061129890000000001, 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718898+0000", 2024-09-14T06:59:34.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718898+0000", 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718898+0000", 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4499999999999997e-07 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718906+0000", 2024-09-14T06:59:34.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5449999999999998e-06 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718915+0000", 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9060000000000008e-06 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718921+0000", 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7859999999999999e-06 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723337+0000", 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044150600000000002 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723383+0000", 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6288000000000001e-05 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723471+0000", 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7967000000000007e-05 2024-09-14T06:59:34.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724604+0000", 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001133656 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724961+0000", 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000356621 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724990+0000", 2024-09-14T06:59:34.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8520000000000001e-05 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725011+0000", 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1095e-05 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.718932+0000", 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.914310241999999, 2024-09-14T06:59:34.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006342421, 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718932+0000", 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718932+0000", 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.718932+0000", 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T06:59:34.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719003+0000", 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999995e-05 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719012+0000", 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5960000000000004e-06 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719018+0000", 2024-09-14T06:59:34.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723736+0000", 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0047176609999999997 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723782+0000", 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6320999999999999e-05 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.723866+0000", 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4053000000000003e-05 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724940+0000", 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010740649999999999 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725226+0000", 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028630000000000002 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725253+0000", 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6849999999999999e-05 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725274+0000", 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0865000000000001e-05 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:34.737 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:14.719029+0000", 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.914212907, 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065583459999999996, 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 76 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719029+0000", 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719029+0000", 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719030+0000", 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999997e-07 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719037+0000", 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3549999999999999e-06 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719044+0000", 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4509999999999997e-06 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.719049+0000", 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724617+0000", 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0055679689999999999 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724658+0000", 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1028000000000001e-05 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.724726+0000", 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8717e-05 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725288+0000", 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00056203299999999998 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725541+0000", 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025287999999999998 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725569+0000", 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7246999999999998e-05 2024-09-14T06:59:34.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.725587+0000", 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8868e-05 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:92 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:15.023225+0000", 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.610016892000001, 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066263060000000002, 2024-09-14T06:59:34.743 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 92 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023225+0000", 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023225+0000", 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023225+0000", 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023228+0000", 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0930000000000001e-06 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023231+0000", 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023234+0000", 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5720000000000001e-06 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025332+0000", 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002098028 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025371+0000", 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9295000000000002e-05 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025436+0000", 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4725000000000001e-05 2024-09-14T06:59:34.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026698+0000", 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001262752 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.029794+0000", 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030956109999999998 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.029823+0000", 2024-09-14T06:59:34.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9064000000000002e-05 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.029851+0000", 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8274999999999999e-05 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:93 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:15.023238+0000", 2024-09-14T06:59:34.751 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.610004236999998, 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077733899999999998, 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 93 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023238+0000", 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023238+0000", 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023238+0000", 2024-09-14T06:59:34.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.28e-07 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023274+0000", 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5649999999999999e-05 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023278+0000", 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0060000000000003e-06 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023280+0000", 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7640000000000001e-06 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025670+0000", 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023893040000000001 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025711+0000", 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1427999999999997e-05 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.025781+0000", 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9717999999999999e-05 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026707+0000", 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00092589699999999996 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.030967+0000", 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042597709999999999 2024-09-14T06:59:34.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.030992+0000", 2024-09-14T06:59:34.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5041999999999998e-05 2024-09-14T06:59:34.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.031011+0000", 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9582e-05 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:94 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:15.023288+0000", 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.609953894, 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010753171000000001, 2024-09-14T06:59:34.758 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 94 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023288+0000", 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023288+0000", 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023288+0000", 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-07 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:34.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023290+0000", 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0439999999999998e-06 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023299+0000", 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8289999999999997e-06 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.023302+0000", 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.005e-06 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026351+0000", 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030484119999999999 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026402+0000", 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1022e-05 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026471+0000", 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9675e-05 2024-09-14T06:59:34.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.026976+0000", 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050500999999999996 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.033999+0000", 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070224730000000004 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.034024+0000", 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4729e-05 2024-09-14T06:59:34.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:15.034041+0000", 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7710000000000002e-05 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:34.764 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:34.766 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:59:34.766 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:34.765+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:34.867 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:34.866+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:34.913 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:34.914 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:34.914 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:34.914 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:34.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:59:34.968 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:34.967+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:35.029 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:35.029 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:35.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:59:35.069 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:35.068+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:35.145 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:35.145 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:35.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:35.145 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:35.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4272.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:20.121031+0000", 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.024151049, 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089234399999999999, 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121031+0000", 2024-09-14T06:59:35.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121031+0000", 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121031+0000", 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4499999999999998e-07 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121038+0000", 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8410000000000002e-06 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121052+0000", 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.382e-05 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121060+0000", 2024-09-14T06:59:35.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8699999999999992e-06 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121121+0000", 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0642e-05 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121144+0000", 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3198999999999999e-05 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:35.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121864+0000", 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072071199999999998 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121923+0000", 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9015000000000003e-05 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.150 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:35.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:59:35.170 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:35.169+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:35.264 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:35.264 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:35.264 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:35.271 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:35.270+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:35.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:59:35.372 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:35.371+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:35.420 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:35.420 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:35.421 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:35.421 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:35.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:59:35.513 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:35.512+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:35.573 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:35.572+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.822733855999999, 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T06:59:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T06:59:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T06:59:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T06:59:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T06:59:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T06:59:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T06:59:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T06:59:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T06:59:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T06:59:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T06:59:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4272.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-14T06:59:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:20.122017+0000", 2024-09-14T06:59:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.454515831, 2024-09-14T06:59:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023148799999999999, 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122017+0000", 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122017+0000", 2024-09-14T06:59:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122017+0000", 2024-09-14T06:59:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7499999999999998e-07 2024-09-14T06:59:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122024+0000", 2024-09-14T06:59:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8750000000000002e-06 2024-09-14T06:59:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122036+0000", 2024-09-14T06:59:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1256e-05 2024-09-14T06:59:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122053+0000", 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7663e-05 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122119+0000", 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5555000000000005e-05 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122128+0000", 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3770000000000002e-06 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122248+0000", 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119787 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:20.125036+0000", 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.45149606, 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019328699999999999, 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T06:59:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125036+0000", 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125036+0000", 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125038+0000", 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125045+0000", 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6510000000000004e-06 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125059+0000", 2024-09-14T06:59:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4093e-05 2024-09-14T06:59:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125065+0000", 2024-09-14T06:59:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6590000000000001e-06 2024-09-14T06:59:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125126+0000", 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0977000000000002e-05 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125230+0000", 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010318 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:20.125374+0000", 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.451158275999999, 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015639699999999999, 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T06:59:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125374+0000", 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125374+0000", 2024-09-14T06:59:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125375+0000", 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7400000000000002e-07 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125381+0000", 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5829999999999997e-06 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125393+0000", 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2316e-05 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125399+0000", 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7540000000000003e-06 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125454+0000", 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5869000000000003e-05 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.125530+0000", 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6001000000000006e-05 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T06:59:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:20.763463+0000", 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.813069437999999, 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045167810000000001, 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763463+0000", 2024-09-14T06:59:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763463+0000", 2024-09-14T06:59:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763465+0000", 2024-09-14T06:59:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1679999999999998e-06 2024-09-14T06:59:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763476+0000", 2024-09-14T06:59:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0883000000000001e-05 2024-09-14T06:59:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763494+0000", 2024-09-14T06:59:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8233999999999999e-05 2024-09-14T06:59:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763503+0000", 2024-09-14T06:59:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0489999999999993e-06 2024-09-14T06:59:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763572+0000", 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8830000000000003e-05 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763704+0000", 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132399 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T06:59:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763821+0000", 2024-09-14T06:59:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011696000000000001 2024-09-14T06:59:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.764623+0000", 2024-09-14T06:59:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080120500000000004 2024-09-14T06:59:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767909+0000", 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003285966 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767947+0000", 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8269999999999998e-05 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767980+0000", 2024-09-14T06:59:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2817000000000001e-05 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4272.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:20.767657+0000", 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.808874982000001, 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033010599999999999, 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767657+0000", 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767657+0000", 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767659+0000", 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767676+0000", 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6365999999999999e-05 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767707+0000", 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1205999999999999e-05 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767728+0000", 2024-09-14T06:59:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1160000000000001e-05 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767899+0000", 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017072100000000001 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767905+0000", 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3600000000000001e-06 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767987+0000", 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999999e-05 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:21.768033+0000", 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.808499292, 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001461525, 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 7 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.768033+0000", 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.768033+0000", 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.768035+0000", 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.768042+0000", 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6960000000000005e-06 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.768057+0000", 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4144999999999999e-05 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.768067+0000", 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0125e-05 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.768134+0000", 2024-09-14T06:59:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6874999999999999e-05 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.768204+0000", 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0772000000000006e-05 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.768317+0000", 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112732 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769039+0000", 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072177700000000005 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769425+0000", 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000385628 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769463+0000", 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8016999999999999e-05 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769495+0000", 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2030999999999999e-05 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4272.0:7 2.2 e22/19 ondisk, result = 0)", 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:21.769340+0000", 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.807191928, 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000161118, 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769340+0000", 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769340+0000", 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769341+0000", 2024-09-14T06:59:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769347+0000", 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1680000000000003e-06 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769357+0000", 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0251000000000001e-05 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769364+0000", 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0829999999999998e-06 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769412+0000", 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8340999999999997e-05 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769418+0000", 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9750000000000004e-06 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:21.769502+0000", 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3327999999999999e-05 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:12 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.812434+0000", 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7640985149999997, 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018126419999999999, 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12 2024-09-14T06:59:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812434+0000", 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812434+0000", 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812435+0000", 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812441+0000", 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2060000000000004e-06 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812451+0000", 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5689999999999994e-06 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812458+0000", 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8349999999999998e-06 2024-09-14T06:59:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812525+0000", 2024-09-14T06:59:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6879999999999997e-05 2024-09-14T06:59:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812708+0000", 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018373599999999999 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812981+0000", 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000272434 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.813665+0000", 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068414099999999998 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814178+0000", 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051320000000000001 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814215+0000", 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6949999999999997e-05 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814246+0000", 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1309999999999997e-05 2024-09-14T06:59:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:13 3.d 3:b701f9ef:::rbd_object_map.114f4f9a926b:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.814384+0000", 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7621479650000005, 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001354402, 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814384+0000", 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814384+0000", 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814386+0000", 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1340000000000001e-06 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814392+0000", 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0549999999999996e-06 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814406+0000", 2024-09-14T06:59:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4691e-05 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814415+0000", 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3499999999999997e-06 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814473+0000", 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8505999999999999e-05 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814558+0000", 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4573000000000005e-05 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814682+0000", 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000124566 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815352+0000", 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066957400000000004 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815634+0000", 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028181700000000002 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815688+0000", 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4256999999999999e-05 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815739+0000", 2024-09-14T06:59:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0878999999999999e-05 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4431.0:13 3.d e27/24 ondisk, result = 0)", 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.815538+0000", 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7609946900000004, 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000209526, 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815538+0000", 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815538+0000", 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815539+0000", 2024-09-14T06:59:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815544+0000", 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1819999999999997e-06 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815556+0000", 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2429000000000001e-05 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815564+0000", 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6179999999999997e-06 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815618+0000", 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3947999999999999e-05 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815627+0000", 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1300000000000007e-06 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.815747+0000", 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012022 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:16 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.819626+0000", 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7569062679999998, 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035476799999999999, 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.819626+0000", 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.819626+0000", 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.819628+0000", 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0439999999999998e-06 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.819638+0000", 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7759999999999992e-06 2024-09-14T06:59:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.819656+0000", 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8141000000000001e-05 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.819668+0000", 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1969e-05 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.819733+0000", 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5320999999999993e-05 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.819801+0000", 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8067999999999999e-05 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.819981+0000", 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017944900000000001 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.820073+0000", 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7564598369999995, 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000199674, 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820073+0000", 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820073+0000", 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820073+0000", 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.06e-07 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820077+0000", 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8199999999999998e-06 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820087+0000", 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8679999999999994e-06 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820094+0000", 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7449999999999998e-06 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820138+0000", 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3964999999999997e-05 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820188+0000", 2024-09-14T06:59:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0383999999999997e-05 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820272+0000", 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4185999999999997e-05 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.820355+0000", 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7561770289999998, 2024-09-14T06:59:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019011500000000001, 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820355+0000", 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820355+0000", 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820356+0000", 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3099999999999997e-07 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820360+0000", 2024-09-14T06:59:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2019999999999997e-06 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820369+0000", 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0310000000000004e-06 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820376+0000", 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.438e-06 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820411+0000", 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5567e-05 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820453+0000", 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1612000000000001e-05 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820545+0000", 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2633999999999994e-05 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:19 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.820625+0000", 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7559069540000003, 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000185458, 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 19 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820625+0000", 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820625+0000", 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820626+0000", 2024-09-14T06:59:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5499999999999997e-07 2024-09-14T06:59:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820631+0000", 2024-09-14T06:59:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4739999999999999e-06 2024-09-14T06:59:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820642+0000", 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1436e-05 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820648+0000", 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2829999999999998e-06 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820689+0000", 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0522000000000003e-05 2024-09-14T06:59:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820735+0000", 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6143999999999997e-05 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.820811+0000", 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5844000000000001e-05 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:20 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.821194+0000", 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7553382519999996, 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033861399999999999, 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821194+0000", 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821194+0000", 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821195+0000", 2024-09-14T06:59:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2900000000000003e-07 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821198+0000", 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.613e-06 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821207+0000", 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5779999999999999e-06 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821213+0000", 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4500000000000001e-06 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821252+0000", 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8373e-05 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821306+0000", 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821533+0000", 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000227161 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:21 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.821629+0000", 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7549029059999999, 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030099700000000001, 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821629+0000", 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821629+0000", 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821630+0000", 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4799999999999998e-07 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821640+0000", 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821649+0000", 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8820000000000007e-06 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821656+0000", 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.365e-06 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821701+0000", 2024-09-14T06:59:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5138e-05 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821750+0000", 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8987999999999999e-05 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.821930+0000", 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018075799999999999 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:22 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.822026+0000", 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7545060350000004, 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020132400000000001, 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 22 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822026+0000", 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822026+0000", 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822027+0000", 2024-09-14T06:59:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1600000000000001e-07 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822030+0000", 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9699999999999999e-06 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822039+0000", 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6090000000000001e-06 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822054+0000", 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5478999999999999e-05 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822092+0000", 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8129e-05 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822136+0000", 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.35e-05 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822228+0000", 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1921000000000003e-05 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:24 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.822705+0000", 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.753827373, 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000181208, 2024-09-14T06:59:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 24 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822705+0000", 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822705+0000", 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822706+0000", 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0500000000000003e-07 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822709+0000", 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8e-06 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822719+0000", 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4180000000000003e-06 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822725+0000", 2024-09-14T06:59:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.489e-06 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822763+0000", 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7836999999999997e-05 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822807+0000", 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3708999999999998e-05 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.822886+0000", 2024-09-14T06:59:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9250000000000002e-05 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:25 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [watch watch cookie 140019960493712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.823035+0000", 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7534970430000003, 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012587259999999999, 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 25 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823035+0000", 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823035+0000", 2024-09-14T06:59:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823036+0000", 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9999999999999997e-07 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823040+0000", 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9049999999999999e-06 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823050+0000", 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0054000000000001e-05 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823056+0000", 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3439999999999997e-06 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823095+0000", 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8281000000000002e-05 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823139+0000", 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4705e-05 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823234+0000", 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5166000000000003e-05 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.823737+0000", 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050216499999999995 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.824221+0000", 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00048480099999999998 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.824251+0000", 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9921999999999999e-05 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.824294+0000", 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2682999999999998e-05 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:35.685 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:35.686 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:59:35.687 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:35.685+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:35.787 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:35.786+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:35.802 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:35.803 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:35.803 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:35.803 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:35.812 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:59:35.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:59:35.888 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:35.887+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:35.917 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:35.917 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:35.917 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:35.917 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:35.917 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:35.926 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:59:35.988 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:35.988+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:36.030 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:36.030 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:36.030 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:36.030 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:36.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.030 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:12 3.9 e27/24 3:91fcf338:::rbd_header.114f4f9a926b:head v 27'1, mlcod=0'0)", 2024-09-14T06:59:36.030 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:30.813360+0000", 2024-09-14T06:59:36.030 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.2164051379999998, 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000694547, 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.813360+0000", 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.813360+0000", 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.813360+0000", 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.13e-07 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.813373+0000", 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2843000000000001e-05 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.813399+0000", 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6146999999999999e-05 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.813412+0000", 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2965999999999999e-05 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.813542+0000", 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012979699999999999 2024-09-14T06:59:36.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.813562+0000", 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0547000000000002e-05 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.814018+0000", 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00045551000000000001 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.814054+0000", 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6223999999999997e-05 2024-09-14T06:59:36.034 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:25 3.9 e27/24 3:91fcf338:::rbd_header.114f4f9a926b:head v 27'2, mlcod=0'0)", 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:30.823412+0000", 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.2063522730000003, 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071347200000000004, 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823412+0000", 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823412+0000", 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823414+0000", 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823424+0000", 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8560000000000002e-06 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823449+0000", 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4833000000000001e-05 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823462+0000", 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3273e-05 2024-09-14T06:59:36.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823591+0000", 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012913099999999999 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823643+0000", 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.181e-05 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.824092+0000", 2024-09-14T06:59:36.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00044955599999999999 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.824126+0000", 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.349e-05 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.039 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:36.041 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:59:36.089 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:36.088+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:36.185 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:36.185 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:36.185 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:36.185 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:36.190 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:36.189+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:36.192 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:59:36.291 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:36.290+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:36.294 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:36.295 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:36.295 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:36.295 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:36.295 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:36.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:59:36.303 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:59:36.392 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:36.391+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:36.423 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:36.423 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:36.440 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:36.440 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:47 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'29, mlcod=17'24)", 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.418828+0000", 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.028427873999998, 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001606089, 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.449 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.418828+0000", 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.418828+0000", 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.418828+0000", 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8999999999999996e-07 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.418896+0000", 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7661000000000002e-05 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.418904+0000", 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6209999999999993e-06 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.418909+0000", 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3669999999999999e-06 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.418950+0000", 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0884999999999999e-05 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.418962+0000", 2024-09-14T06:59:36.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2319e-05 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420392+0000", 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001430387 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420434+0000", 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1158999999999997e-05 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'26)", 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.419646+0000", 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.027609866999999, 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001682153, 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.419646+0000", 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.419646+0000", 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.419646+0000", 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9999999999999997e-07 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.419665+0000", 2024-09-14T06:59:36.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8612000000000001e-05 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.419674+0000", 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5689999999999994e-06 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.419685+0000", 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0248000000000001e-05 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.419736+0000", 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1038000000000002e-05 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.419748+0000", 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.224e-05 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.421294+0000", 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00154627 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.421328+0000", 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3575999999999998e-05 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'26)", 2024-09-14T06:59:36.458 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.420146+0000", 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.027109812999999, 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017583449999999999, 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420146+0000", 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420146+0000", 2024-09-14T06:59:36.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420146+0000", 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2600000000000002e-07 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420151+0000", 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4129999999999999e-06 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420159+0000", 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5960000000000004e-06 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420165+0000", 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5629999999999999e-06 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420208+0000", 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3302e-05 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.420220+0000", 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1977000000000001e-05 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.421860+0000", 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016397849999999999 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.421904+0000", 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4082999999999998e-05 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.462 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:53 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'35, mlcod=17'33)", 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.423281+0000", 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.023974324000001, 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00211137, 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423281+0000", 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423281+0000", 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423282+0000", 2024-09-14T06:59:36.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0699999999999996e-07 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423345+0000", 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3468999999999998e-05 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423358+0000", 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2435999999999999e-05 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423363+0000", 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2850000000000002e-06 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423395+0000", 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1602e-05 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423407+0000", 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2692e-05 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425359+0000", 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019521709999999999 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425392+0000", 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3008e-05 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:54 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-14T06:59:36.467 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.423853+0000", 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.023402652000001, 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0043481910000000004, 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423853+0000", 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423853+0000", 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423853+0000", 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8699999999999995e-07 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423924+0000", 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1045999999999997e-05 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423933+0000", 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.49e-06 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423937+0000", 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8700000000000002e-06 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423978+0000", 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1625000000000002e-05 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423993+0000", 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5104e-05 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.428168+0000", 2024-09-14T06:59:36.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041749129999999997 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.428201+0000", 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2555999999999998e-05 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:55 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.424336+0000", 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.022918894, 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003882861, 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424336+0000", 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424336+0000", 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424337+0000", 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2099999999999997e-07 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424411+0000", 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3936999999999994e-05 2024-09-14T06:59:36.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424421+0000", 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0328e-05 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424426+0000", 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2810000000000002e-06 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424461+0000", 2024-09-14T06:59:36.475 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5244999999999999e-05 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424475+0000", 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4164e-05 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.428204+0000", 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037293119999999998 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.428219+0000", 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5073e-05 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:56 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'38, mlcod=17'33)", 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.424920+0000", 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.022335534, 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061030700000000004, 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.477 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424920+0000", 2024-09-14T06:59:36.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424920+0000", 2024-09-14T06:59:36.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424920+0000", 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3799999999999997e-07 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425021+0000", 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000100388 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425029+0000", 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6219999999999998e-06 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425033+0000", 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0400000000000003e-06 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425082+0000", 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8511999999999999e-05 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425099+0000", 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7354999999999999e-05 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.430989+0000", 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0058901739999999998 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.431023+0000", 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3441000000000001e-05 2024-09-14T06:59:36.481 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:57 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'39, mlcod=17'37)", 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.431965+0000", 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.015290132000001, 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017434289999999999, 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.431965+0000", 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.431965+0000", 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.431966+0000", 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0100000000000004e-07 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.431971+0000", 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8670000000000002e-06 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.431982+0000", 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1523e-05 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.431991+0000", 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4440000000000008e-06 2024-09-14T06:59:36.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.432036+0000", 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4802000000000002e-05 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.432049+0000", 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3856000000000001e-05 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.433673+0000", 2024-09-14T06:59:36.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016240009999999999 2024-09-14T06:59:36.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.433709+0000", 2024-09-14T06:59:36.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5234999999999997e-05 2024-09-14T06:59:36.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.486 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'40, mlcod=17'38)", 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.434359+0000", 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.012896205000001, 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002374223, 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434359+0000", 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434359+0000", 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434360+0000", 2024-09-14T06:59:36.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6000000000000003e-07 2024-09-14T06:59:36.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434365+0000", 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0509999999999996e-06 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434375+0000", 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0165e-05 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434381+0000", 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1380000000000004e-06 2024-09-14T06:59:36.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434458+0000", 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6414000000000004e-05 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434472+0000", 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4732999999999999e-05 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.436701+0000", 2024-09-14T06:59:36.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022286739999999999 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.436733+0000", 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2388000000000002e-05 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.719839+0000", 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.727416587, 2024-09-14T06:59:36.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019483739999999999, 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.719839+0000", 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.719839+0000", 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.719840+0000", 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3e-07 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.719845+0000", 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2580000000000003e-06 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.719857+0000", 2024-09-14T06:59:36.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1795e-05 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.719863+0000", 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2439999999999998e-06 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.719920+0000", 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7433999999999999e-05 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.719932+0000", 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1440000000000001e-05 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721757+0000", 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001825545 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721787+0000", 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9927999999999999e-05 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.496 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'42)", 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.720831+0000", 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.726424693999999, 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014515559999999999, 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.720831+0000", 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.720831+0000", 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.720831+0000", 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5300000000000003e-07 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.720893+0000", 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2034999999999998e-05 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.720903+0000", 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2399999999999996e-06 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.720909+0000", 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8769999999999998e-06 2024-09-14T06:59:36.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.720945+0000", 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6062000000000001e-05 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.720958+0000", 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3159000000000001e-05 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722244+0000", 2024-09-14T06:59:36.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012860090000000001 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722282+0000", 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8420999999999997e-05 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:71 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'49, mlcod=17'43)", 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.721844+0000", 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.725411319999999, 2024-09-14T06:59:36.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014381730000000001, 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721844+0000", 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721844+0000", 2024-09-14T06:59:36.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721845+0000", 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5400000000000001e-07 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721852+0000", 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2520000000000002e-06 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721861+0000", 2024-09-14T06:59:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8640000000000002e-06 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721866+0000", 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2560000000000002e-06 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721903+0000", 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6531e-05 2024-09-14T06:59:36.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.721915+0000", 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2026e-05 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.723248+0000", 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0013328999999999999 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.723282+0000", 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4690000000000002e-05 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'43)", 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.722244+0000", 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.725011619, 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001599851, 2024-09-14T06:59:36.506 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722244+0000", 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722244+0000", 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722244+0000", 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8100000000000002e-07 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722304+0000", 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9494000000000003e-05 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722315+0000", 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0645999999999999e-05 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722321+0000", 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2500000000000003e-06 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722359+0000", 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8438e-05 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.722376+0000", 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6239000000000002e-05 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.723804+0000", 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014286819999999999 2024-09-14T06:59:36.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.723844+0000", 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9421000000000001e-05 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:92 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'66, mlcod=18'63)", 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.025527+0000", 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.421728539, 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.004154062, 2024-09-14T06:59:36.511 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025527+0000", 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025527+0000", 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025527+0000", 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025531+0000", 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0899999999999998e-06 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025541+0000", 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7729999999999994e-06 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025547+0000", 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025579+0000", 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.269e-05 2024-09-14T06:59:36.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025592+0000", 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2799000000000001e-05 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.029639+0000", 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040465079999999999 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.029681+0000", 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2333000000000003e-05 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:93 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'67, mlcod=18'63)", 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.025869+0000", 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.421386321, 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0050062630000000004, 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.516 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025869+0000", 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025869+0000", 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025870+0000", 2024-09-14T06:59:36.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1600000000000001e-07 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025932+0000", 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2438000000000001e-05 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025939+0000", 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025944+0000", 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0150000000000003e-06 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025970+0000", 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5500999999999999e-05 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025981+0000", 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0925e-05 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.030837+0000", 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048562249999999996 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.030875+0000", 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8488999999999997e-05 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.520 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:94 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'68, mlcod=18'64)", 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.026545+0000", 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.420710454999998, 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0073334020000000001, 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026545+0000", 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026545+0000", 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026545+0000", 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.01e-07 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026551+0000", 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2510000000000002e-06 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026557+0000", 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0639999999999999e-06 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026561+0000", 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3819999999999997e-06 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.029702+0000", 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031409540000000001 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.029717+0000", 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5029e-05 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.033841+0000", 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041235330000000004 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.033878+0000", 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7787999999999998e-05 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.525 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:95 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'69, mlcod=18'67)", 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.034664+0000", 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.412591862999999, 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015323019999999999, 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.034664+0000", 2024-09-14T06:59:36.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.034664+0000", 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.034664+0000", 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5400000000000001e-07 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.034715+0000", 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0408999999999998e-05 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.034724+0000", 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.893e-06 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.034730+0000", 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9689999999999999e-06 2024-09-14T06:59:36.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.034773+0000", 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2848e-05 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.034800+0000", 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6452e-05 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.036162+0000", 2024-09-14T06:59:36.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001362395 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.036196+0000", 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3682e-05 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:96 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'70, mlcod=18'67)", 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.035205+0000", 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.412049984999999, 2024-09-14T06:59:36.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0036047890000000002, 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035205+0000", 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035205+0000", 2024-09-14T06:59:36.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035206+0000", 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6199999999999999e-07 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035256+0000", 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9895999999999998e-05 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035263+0000", 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1069999999999999e-06 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035266+0000", 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.36e-06 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035305+0000", 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9110999999999999e-05 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035323+0000", 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8029e-05 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.038769+0000", 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034452549999999999 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.038810+0000", 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1668999999999997e-05 2024-09-14T06:59:36.534 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:98 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'72, mlcod=18'70)", 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.042159+0000", 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 21.405096582999999, 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002029229, 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042159+0000", 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042159+0000", 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042159+0000", 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8899999999999999e-07 2024-09-14T06:59:36.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042163+0000", 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9859999999999996e-06 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042173+0000", 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.73e-06 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042178+0000", 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.605e-06 2024-09-14T06:59:36.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042219+0000", 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1415000000000002e-05 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042239+0000", 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0072999999999999e-05 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.044096+0000", 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018567989999999999 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.044188+0000", 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2032000000000002e-05 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:30.810150+0000", 2024-09-14T06:59:36.539 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.6371056309999998, 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001687233, 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 11 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.810150+0000", 2024-09-14T06:59:36.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.810150+0000", 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.810151+0000", 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.173e-06 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:36.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.810157+0000", 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6579999999999999e-06 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.810168+0000", 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.094e-05 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.810175+0000", 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3849999999999996e-06 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.810214+0000", 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8825999999999997e-05 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.810324+0000", 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000110612 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.810450+0000", 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012607100000000001 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.811015+0000", 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00056450399999999996 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.811771+0000", 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00075569799999999996 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.811810+0000", 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8983999999999998e-05 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:36.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:36.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:36.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.811837+0000", 2024-09-14T06:59:36.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7382e-05 2024-09-14T06:59:36.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.545 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:36.545 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:36.545 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:36.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:36.546 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:36.547 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:36.547 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:36.548 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:59:36.548 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:36.546+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:36.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:36.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:36.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:36.550 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:36.550 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:36.551 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:36.648 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:36.647+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:36.691 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:36.691 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:36.699 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:59:36.748 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:36.747+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:36.803 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:36.803 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:36.813 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:59:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T06:59:36.854 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:36.853+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:36.949 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:36.949+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:36.956 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:36.956 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:36.965 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:59:37.050 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:37.050+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:37.112 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:37.112 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:37.120 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:59:37.152 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.164 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:37.164+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:37.178 INFO:teuthology.orchestra.run.smithi007.stdout:137 writesame 0xd244c50 thru 0xd2466f1 data_size 0x7(0x1aa2 bytes) 2024-09-14T06:59:37.183 INFO:teuthology.orchestra.run.smithi007.stdout:138 trunc from 0xd2466f2 to 0x9da3d75 2024-09-14T06:59:37.198 INFO:teuthology.orchestra.run.smithi007.stdout:140 read 0x637fa02 thru 0x63843d0 (0x49cf bytes) 2024-09-14T06:59:37.199 INFO:teuthology.orchestra.run.smithi007.stdout:141 read 0x943b08c thru 0x9440d81 (0x5cf6 bytes) 2024-09-14T06:59:37.199 INFO:teuthology.orchestra.run.smithi007.stdout:142 write 0x352acd4 thru 0x352eec9 (0x41f6 bytes) 2024-09-14T06:59:37.201 INFO:teuthology.orchestra.run.smithi007.stdout:143 write 0x91a80fb thru 0x91ac305 (0x420b bytes) 2024-09-14T06:59:37.202 INFO:teuthology.orchestra.run.smithi007.stdout:145 writesame 0x6a0e93c thru 0x6a18075 data_size 0xd(0x973a bytes) 2024-09-14T06:59:37.213 INFO:teuthology.orchestra.run.smithi007.stdout:146 writesame 0xa2e83a7 thru 0xa2ef8c8 data_size 0xb(0x7522 bytes) 2024-09-14T06:59:37.213 INFO:teuthology.orchestra.run.smithi007.stdout:147 read 0x69bb5e3 thru 0x69ca470 (0xee8e bytes) 2024-09-14T06:59:37.213 INFO:teuthology.orchestra.run.smithi007.stdout:148 read 0x3fae45e thru 0x3fb96e8 (0xb28b bytes) 2024-09-14T06:59:37.213 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.213 INFO:teuthology.orchestra.run.smithi007.stdout:152 read 0x1ca6ca1 thru 0x1cb52c5 (0xe625 bytes) 2024-09-14T06:59:37.220 INFO:teuthology.orchestra.run.smithi007.stdout:153 trunc from 0xa2ef8c9 to 0x36d577f 2024-09-14T06:59:37.231 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:37.232 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:37.241 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:59:37.244 INFO:teuthology.orchestra.run.smithi007.stdout:154 writesame 0xc155cfc thru 0xc1657fe data_size 0x1(0xfb03 bytes) 2024-09-14T06:59:37.244 INFO:teuthology.orchestra.run.smithi007.stdout:155 read 0x10cd60b thru 0x10d2850 (0x5246 bytes) 2024-09-14T06:59:37.244 INFO:teuthology.orchestra.run.smithi007.stdout:157 punch from 0x5ab8485 to 0x5ac6809, (0xe384 bytes) 2024-09-14T06:59:37.251 INFO:teuthology.orchestra.run.smithi007.stdout:158 trunc from 0xc1657ff to 0x5f4040f 2024-09-14T06:59:37.258 INFO:teuthology.orchestra.run.smithi007.stdout:160 punch from 0x3e1924f to 0x3e239ba, (0xa76b bytes) 2024-09-14T06:59:37.259 INFO:teuthology.orchestra.run.smithi007.stdout:161 write 0x5d7320d thru 0x5d8018c (0xcf80 bytes) 2024-09-14T06:59:37.260 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.260 INFO:teuthology.orchestra.run.smithi007.stdout:163 write 0xa05b49 thru 0xa0e96c (0x8e24 bytes) 2024-09-14T06:59:37.262 INFO:teuthology.orchestra.run.smithi007.stdout:165 read 0x1bb4558 thru 0x1bc2d40 (0xe7e9 bytes) 2024-09-14T06:59:37.262 INFO:teuthology.orchestra.run.smithi007.stdout:166 read 0x30b208d thru 0x30b3519 (0x148d bytes) 2024-09-14T06:59:37.278 INFO:teuthology.orchestra.run.smithi007.stdout:167 writesame 0xeaa802e thru 0xeab2d4e data_size 0x29(0xad21 bytes) 2024-09-14T06:59:37.279 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.283 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:37.282+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:37.284 INFO:teuthology.orchestra.run.smithi007.stdout:169 trunc from 0xeab2d4f to 0x9577f4f 2024-09-14T06:59:37.291 INFO:teuthology.orchestra.run.smithi007.stdout:170 read 0x804b711 thru 0x804f4f0 (0x3de0 bytes) 2024-09-14T06:59:37.292 INFO:teuthology.orchestra.run.smithi007.stdout:172 punch from 0x6b16ea1 to 0x6b17316, (0x475 bytes) 2024-09-14T06:59:37.298 INFO:teuthology.orchestra.run.smithi007.stdout:174 writesame 0xa336df2 thru 0xa346459 data_size 0xa6(0xf668 bytes) 2024-09-14T06:59:37.299 INFO:teuthology.orchestra.run.smithi007.stdout:175 write 0x564f66 thru 0x5694fa (0x4595 bytes) 2024-09-14T06:59:37.301 INFO:teuthology.orchestra.run.smithi007.stdout:176 write 0x6c06632 thru 0x6c08a44 (0x2413 bytes) 2024-09-14T06:59:37.302 INFO:teuthology.orchestra.run.smithi007.stdout:178 read 0x8f5e26 thru 0x8fe7b2 (0x898d bytes) 2024-09-14T06:59:37.303 INFO:teuthology.orchestra.run.smithi007.stdout:179 punch from 0x643a47d to 0x6447262, (0xcde5 bytes) 2024-09-14T06:59:37.303 INFO:teuthology.orchestra.run.smithi007.stdout:180 punch from 0x7f9dd9e to 0x7faa715, (0xc977 bytes) 2024-09-14T06:59:37.303 INFO:teuthology.orchestra.run.smithi007.stdout:181 read 0x4bf0504 thru 0x4bfdc86 (0xd783 bytes) 2024-09-14T06:59:37.313 INFO:teuthology.orchestra.run.smithi007.stdout:183 write 0xdd7c007 thru 0xdd89e00 (0xddfa bytes) 2024-09-14T06:59:37.322 INFO:teuthology.orchestra.run.smithi007.stdout:184 write 0xe78ccba thru 0xe799641 (0xc988 bytes) 2024-09-14T06:59:37.326 INFO:teuthology.orchestra.run.smithi007.stdout:185 writesame 0x29bcb17 thru 0x29c7275 data_size 0x1(0xa75f bytes) 2024-09-14T06:59:37.348 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.348 INFO:teuthology.orchestra.run.smithi007.stdout:187 read 0x248e29f thru 0x2496a73 (0x87d5 bytes) 2024-09-14T06:59:37.348 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.348 INFO:teuthology.orchestra.run.smithi007.stdout:192 read 0xb8f97b1 thru 0xb8fd9b9 (0x4209 bytes) 2024-09-14T06:59:37.348 INFO:teuthology.orchestra.run.smithi007.stdout:193 write 0x1ce738 thru 0x1d81b8 (0x9a81 bytes) 2024-09-14T06:59:37.350 INFO:teuthology.orchestra.run.smithi007.stdout:194 punch from 0x57a76b4 to 0x57ad75a, (0x60a6 bytes) 2024-09-14T06:59:37.350 INFO:teuthology.orchestra.run.smithi007.stdout:196 writesame 0x79293f thru 0x798cf9 data_size 0xb(0x63bb bytes) 2024-09-14T06:59:37.351 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.352 INFO:teuthology.orchestra.run.smithi007.stdout:199 write 0x9bcbc46 thru 0x9bd04a4 (0x485f bytes) 2024-09-14T06:59:37.352 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:37.352+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:37.358 INFO:teuthology.orchestra.run.smithi007.stdout:201 trunc from 0xe799642 to 0x9d13e00 2024-09-14T06:59:37.372 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.372 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.372 INFO:teuthology.orchestra.run.smithi007.stdout:206 punch from 0x50d49c6 to 0x50d8b4b, (0x4185 bytes) 2024-09-14T06:59:37.383 INFO:teuthology.orchestra.run.smithi007.stdout:207 write 0xcca47c7 thru 0xccaf510 (0xad4a bytes) 2024-09-14T06:59:37.386 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.386 INFO:teuthology.orchestra.run.smithi007.stdout:211 read 0x6edb3e1 thru 0x6ee4016 (0x8c36 bytes) 2024-09-14T06:59:37.386 INFO:teuthology.orchestra.run.smithi007.stdout:212 read 0x8bba605 thru 0x8bbebfe (0x45fa bytes) 2024-09-14T06:59:37.387 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.387 INFO:teuthology.orchestra.run.smithi007.stdout:215 writesame 0x3477b1f thru 0x347b36e data_size 0x44(0x3850 bytes) 2024-09-14T06:59:37.388 INFO:teuthology.orchestra.run.smithi007.stdout:216 write 0xb93fc20 thru 0xb943176 (0x3557 bytes) 2024-09-14T06:59:37.392 INFO:teuthology.orchestra.run.smithi007.stdout:218 punch from 0xa3aa0c7 to 0xa3b02a9, (0x61e2 bytes) 2024-09-14T06:59:37.392 INFO:teuthology.orchestra.run.smithi007.stdout:219 writesame 0xa935716 thru 0xa944305 data_size 0x4(0xebf0 bytes) 2024-09-14T06:59:37.398 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:37.398 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:37.400 INFO:teuthology.orchestra.run.smithi007.stdout:220 write 0x466a260 thru 0x466b4ee (0x128f bytes) 2024-09-14T06:59:37.401 INFO:teuthology.orchestra.run.smithi007.stdout:221 read 0x89343ef thru 0x8936856 (0x2468 bytes) 2024-09-14T06:59:37.401 INFO:teuthology.orchestra.run.smithi007.stdout:222 read 0xee3f66 thru 0xee4b1f (0xbba bytes) 2024-09-14T06:59:37.402 INFO:teuthology.orchestra.run.smithi007.stdout:223 punch from 0x971051e to 0x9716d6a, (0x684c bytes) 2024-09-14T06:59:37.402 INFO:teuthology.orchestra.run.smithi007.stdout:224 punch from 0x718db31 to 0x7190bdc, (0x30ab bytes) 2024-09-14T06:59:37.402 INFO:teuthology.orchestra.run.smithi007.stdout:225 write 0xa062a47 thru 0xa069e74 (0x742e bytes) 2024-09-14T06:59:37.405 INFO:teuthology.orchestra.run.smithi007.stdout:228 trunc from 0xccaf511 to 0xe5b86ba 2024-09-14T06:59:37.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:59:37.412 INFO:teuthology.orchestra.run.smithi007.stdout:230 read 0x2e053f8 thru 0x2e082ba (0x2ec3 bytes) 2024-09-14T06:59:37.412 INFO:teuthology.orchestra.run.smithi007.stdout:233 write 0xda8ef2c thru 0xda990a4 (0xa179 bytes) 2024-09-14T06:59:37.418 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.418 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.418 INFO:teuthology.orchestra.run.smithi007.stdout:236 read 0x6813a98 thru 0x681b73c (0x7ca5 bytes) 2024-09-14T06:59:37.419 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.422 INFO:teuthology.orchestra.run.smithi007.stdout:241 trunc from 0xe5b86ba to 0xb793c09 2024-09-14T06:59:37.441 INFO:teuthology.orchestra.run.smithi007.stdout:243 read 0x365720a thru 0x365cbf8 (0x59ef bytes) 2024-09-14T06:59:37.449 INFO:teuthology.orchestra.run.smithi007.stdout:244 writesame 0xd0279a2 thru 0xd02f8ce data_size 0x1(0x7f2d bytes) 2024-09-14T06:59:37.449 INFO:teuthology.orchestra.run.smithi007.stdout:245 write 0xa34bdc2 thru 0xa34ead5 (0x2d14 bytes) 2024-09-14T06:59:37.450 INFO:teuthology.orchestra.run.smithi007.stdout:246 writesame 0xb560fa2 thru 0xb566646 data_size 0x1(0x56a5 bytes) 2024-09-14T06:59:37.453 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:37.453+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:37.460 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:37.461 INFO:teuthology.orchestra.run.smithi007.stdout:249 write 0x6ceda37 thru 0x6cfc801 (0xedcb bytes) 2024-09-14T06:59:37.462 INFO:teuthology.orchestra.run.smithi007.stdout:250 read 0xc811693 thru 0xc81a952 (0x92c0 bytes) 2024-09-14T06:59:37.462 INFO:teuthology.orchestra.run.smithi007.stdout:251 punch from 0x918e6e4 to 0x91926f0, (0x400c bytes) 2024-09-14T06:59:37.463 INFO:teuthology.orchestra.run.smithi007.stdout:252 read 0x15af257 thru 0x15b0d01 (0x1aab bytes) 2024-09-14T06:59:37.474 INFO:teuthology.orchestra.run.smithi007.stdout:253 trunc from 0xd02f8cf to 0x2c218ab 2024-09-14T06:59:37.485 INFO:teuthology.orchestra.run.smithi007.stdout:254 punch from 0xcc0338 to 0xccf391, (0xf059 bytes) 2024-09-14T06:59:37.495 INFO:teuthology.orchestra.run.smithi007.stdout:255 trunc from 0x2c218ab to 0xc06e183 2024-09-14T06:59:37.502 INFO:teuthology.orchestra.run.smithi007.stdout:256 writesame 0x9b56860 thru 0x9b662f5 data_size 0x19(0xfa96 bytes) 2024-09-14T06:59:37.504 INFO:teuthology.orchestra.run.smithi007.stdout:257 clone 2 order 25 su 1048576 sc 10 2024-09-14T06:59:37.526 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:37.526 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:37.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:59:37.577 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:37.576+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:37.656 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:37.655+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:37.683 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:37.683 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:37.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:59:37.757 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:37.756+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:37.798 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:37.799 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:37.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:59:37.858 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:37.857+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:37.897 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone1 from 0xc06e183 (overlap 0x2c218ab) to 0x0 2024-09-14T06:59:37.922 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:37.923 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:37.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:59:37.952 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-14T06:59:37.959 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:37.958+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:38.018 INFO:teuthology.orchestra.run.smithi007.stdout:260 trunc from 0xc06e183 to 0x9060679 2024-09-14T06:59:38.048 INFO:teuthology.orchestra.run.smithi007.stdout:261 writesame 0x81222a thru 0x822097 data_size 0x1d(0xfe6e bytes) 2024-09-14T06:59:38.053 INFO:teuthology.orchestra.run.smithi007.stdout:262 trunc from 0x9060679 to 0xaf6e235 2024-09-14T06:59:38.058 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.058 INFO:teuthology.orchestra.run.smithi007.stdout:267 write 0x8d06c9e thru 0x8d10df3 (0xa156 bytes) 2024-09-14T06:59:38.059 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:38.058+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:38.066 INFO:teuthology.orchestra.run.smithi007.stdout:268 write 0xc02ef33 thru 0xc0369da (0x7aa8 bytes) 2024-09-14T06:59:38.070 INFO:teuthology.orchestra.run.smithi007.stdout:270 trunc from 0xc0369db to 0x9a9a307 2024-09-14T06:59:38.078 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:38.078 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:38.088 INFO:teuthology.orchestra.run.smithi007.stdout:272 writesame 0xe182249 thru 0xe1849df data_size 0x1(0x2797 bytes) 2024-09-14T06:59:38.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:59:38.096 INFO:teuthology.orchestra.run.smithi007.stdout:273 writesame 0xe9a3263 thru 0xe9b2b5d data_size 0x1(0xf8fb bytes) 2024-09-14T06:59:38.098 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.111 INFO:teuthology.orchestra.run.smithi007.stdout:277 trunc from 0xe9b2b5e to 0x254035e 2024-09-14T06:59:38.120 INFO:teuthology.orchestra.run.smithi007.stdout:278 trunc from 0x254035e to 0xbe885d 2024-09-14T06:59:38.140 INFO:teuthology.orchestra.run.smithi007.stdout:280 write 0x74c52e3 thru 0x74c9d87 (0x4aa5 bytes) 2024-09-14T06:59:38.155 INFO:teuthology.orchestra.run.smithi007.stdout:281 write 0xe99384c thru 0xe99ae00 (0x75b5 bytes) 2024-09-14T06:59:38.156 INFO:teuthology.orchestra.run.smithi007.stdout:282 punch from 0x9a81712 to 0x9a8a753, (0x9041 bytes) 2024-09-14T06:59:38.156 INFO:teuthology.orchestra.run.smithi007.stdout:284 writesame 0x548fa7 thru 0x55745d data_size 0x1d(0xe4b7 bytes) 2024-09-14T06:59:38.158 INFO:teuthology.orchestra.run.smithi007.stdout:285 writesame 0x2f1b934 thru 0x2f1d85e data_size 0x1(0x1f2b bytes) 2024-09-14T06:59:38.160 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:38.159+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:38.162 INFO:teuthology.orchestra.run.smithi007.stdout:286 write 0x851c9af thru 0x8529a33 (0xd085 bytes) 2024-09-14T06:59:38.164 INFO:teuthology.orchestra.run.smithi007.stdout:287 read 0x5a2f0f4 thru 0x5a2fdbd (0xcca bytes) 2024-09-14T06:59:38.164 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.164 INFO:teuthology.orchestra.run.smithi007.stdout:289 punch from 0x1597cf5 to 0x15a2d7b, (0xb086 bytes) 2024-09-14T06:59:38.164 INFO:teuthology.orchestra.run.smithi007.stdout:292 read 0x5e66fb1 thru 0x5e7238f (0xb3df bytes) 2024-09-14T06:59:38.165 INFO:teuthology.orchestra.run.smithi007.stdout:293 write 0x283cbcd thru 0x2840cdd (0x4111 bytes) 2024-09-14T06:59:38.166 INFO:teuthology.orchestra.run.smithi007.stdout:294 write 0x313aaf3 thru 0x313de52 (0x3360 bytes) 2024-09-14T06:59:38.171 INFO:teuthology.orchestra.run.smithi007.stdout:295 trunc from 0xe99ae01 to 0xc48374c 2024-09-14T06:59:38.178 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.178 INFO:teuthology.orchestra.run.smithi007.stdout:297 write 0x7fe3d98 thru 0x7ff2490 (0xe6f9 bytes) 2024-09-14T06:59:38.180 INFO:teuthology.orchestra.run.smithi007.stdout:298 punch from 0x1c4f72e to 0x1c54aad, (0x537f bytes) 2024-09-14T06:59:38.181 INFO:teuthology.orchestra.run.smithi007.stdout:299 punch from 0x579775f to 0x57a5e5f, (0xe700 bytes) 2024-09-14T06:59:38.188 INFO:teuthology.orchestra.run.smithi007.stdout:300 writesame 0xe435451 thru 0xe4400c1 data_size 0x6d(0xac71 bytes) 2024-09-14T06:59:38.198 INFO:teuthology.orchestra.run.smithi007.stdout:301 trunc from 0xe4400c2 to 0x699c9ac 2024-09-14T06:59:38.206 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.214 INFO:teuthology.orchestra.run.smithi007.stdout:306 write 0x901b242 thru 0x901bffd (0xdbc bytes) 2024-09-14T06:59:38.219 INFO:teuthology.orchestra.run.smithi007.stdout:307 trunc from 0x901bffe to 0x588f499 2024-09-14T06:59:38.225 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:38.225 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:38.230 INFO:teuthology.orchestra.run.smithi007.stdout:308 punch from 0x41f888c to 0x4200f4e, (0x86c2 bytes) 2024-09-14T06:59:38.230 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.230 INFO:teuthology.orchestra.run.smithi007.stdout:311 write 0x2b200f0 thru 0x2b2343d (0x334e bytes) 2024-09-14T06:59:38.231 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.231 INFO:teuthology.orchestra.run.smithi007.stdout:313 write 0x1b588f4 thru 0x1b6366a (0xad77 bytes) 2024-09-14T06:59:38.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:59:38.245 INFO:teuthology.orchestra.run.smithi007.stdout:314 write 0xa270bcd thru 0xa274916 (0x3d4a bytes) 2024-09-14T06:59:38.245 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.245 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.245 INFO:teuthology.orchestra.run.smithi007.stdout:318 read 0x8b6df7b thru 0x8b6f4e1 (0x1567 bytes) 2024-09-14T06:59:38.256 INFO:teuthology.orchestra.run.smithi007.stdout:319 write 0xebdd590 thru 0xebe7ac5 (0xa536 bytes) 2024-09-14T06:59:38.257 INFO:teuthology.orchestra.run.smithi007.stdout:320 punch from 0x3ae07d1 to 0x3aee76a, (0xdf99 bytes) 2024-09-14T06:59:38.257 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.257 INFO:teuthology.orchestra.run.smithi007.stdout:323 write 0xac3ea06 thru 0xac44613 (0x5c0e bytes) 2024-09-14T06:59:38.259 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.259 INFO:teuthology.orchestra.run.smithi007.stdout:327 read 0xc70b795 thru 0xc718139 (0xc9a5 bytes) 2024-09-14T06:59:38.259 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.260 INFO:teuthology.orchestra.run.smithi007.stdout:329 read 0xd972370 thru 0xd97b97a (0x960b bytes) 2024-09-14T06:59:38.260 INFO:teuthology.orchestra.run.smithi007.stdout:331 writesame 0x7fcaaeb thru 0x7fd3fc8 data_size 0x67(0x94de bytes) 2024-09-14T06:59:38.260 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:38.260+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:38.261 INFO:teuthology.orchestra.run.smithi007.stdout:333 writesame 0x2eeedfe thru 0x2ef6b1a data_size 0x1(0x7d1d bytes) 2024-09-14T06:59:38.282 INFO:teuthology.orchestra.run.smithi007.stdout:335 trunc from 0xebe7ac6 to 0x98be9ff 2024-09-14T06:59:38.293 INFO:teuthology.orchestra.run.smithi007.stdout:336 read 0x65047db thru 0x6508dec (0x4612 bytes) 2024-09-14T06:59:38.293 INFO:teuthology.orchestra.run.smithi007.stdout:338 write 0x67a450b thru 0x67ad79a (0x9290 bytes) 2024-09-14T06:59:38.299 INFO:teuthology.orchestra.run.smithi007.stdout:339 trunc from 0x98be9ff to 0x5faa6de 2024-09-14T06:59:38.324 INFO:teuthology.orchestra.run.smithi007.stdout:340 write 0xe6d1dfc thru 0xe6db580 (0x9785 bytes) 2024-09-14T06:59:38.325 INFO:teuthology.orchestra.run.smithi007.stdout:341 read 0x2f5fc1d thru 0x2f6b871 (0xbc55 bytes) 2024-09-14T06:59:38.326 INFO:teuthology.orchestra.run.smithi007.stdout:343 read 0xd3f15c1 thru 0xd3f6606 (0x5046 bytes) 2024-09-14T06:59:38.326 INFO:teuthology.orchestra.run.smithi007.stdout:344 writesame 0x71a875d thru 0x71b0096 data_size 0x3b(0x793a bytes) 2024-09-14T06:59:38.328 INFO:teuthology.orchestra.run.smithi007.stdout:347 writesame 0x3eb48e8 thru 0x3eba677 data_size 0x18(0x5d90 bytes) 2024-09-14T06:59:38.329 INFO:teuthology.orchestra.run.smithi007.stdout:348 write 0x584a7ef thru 0x5855af4 (0xb306 bytes) 2024-09-14T06:59:38.331 INFO:teuthology.orchestra.run.smithi007.stdout:351 write 0xa93a8ef thru 0xa93cb8e (0x22a0 bytes) 2024-09-14T06:59:38.333 INFO:teuthology.orchestra.run.smithi007.stdout:353 writesame 0x4aa9746 thru 0x4ab582c data_size 0x5d(0xc0e7 bytes) 2024-09-14T06:59:38.335 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.335 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.335 INFO:teuthology.orchestra.run.smithi007.stdout:356 read 0x396149 thru 0x39f9fb (0x98b3 bytes) 2024-09-14T06:59:38.335 INFO:teuthology.orchestra.run.smithi007.stdout:357 write 0x61be29 thru 0x62ac01 (0xedd9 bytes) 2024-09-14T06:59:38.337 INFO:teuthology.orchestra.run.smithi007.stdout:359 read 0x365b670 thru 0x36638bc (0x824d bytes) 2024-09-14T06:59:38.337 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.337 INFO:teuthology.orchestra.run.smithi007.stdout:363 writesame 0x351353 thru 0x355a8b data_size 0x1(0x4739 bytes) 2024-09-14T06:59:38.345 INFO:teuthology.orchestra.run.smithi007.stdout:364 read 0x4839296 thru 0x48409a5 (0x7710 bytes) 2024-09-14T06:59:38.345 INFO:teuthology.orchestra.run.smithi007.stdout:365 read 0x9bb71f5 thru 0x9bbc8ae (0x56ba bytes) 2024-09-14T06:59:38.346 INFO:teuthology.orchestra.run.smithi007.stdout:366 write 0xe2d9bf2 thru 0xe2e6312 (0xc721 bytes) 2024-09-14T06:59:38.348 INFO:teuthology.orchestra.run.smithi007.stdout:368 writesame 0x1bf4255 thru 0x1bff812 data_size 0x2(0xb5be bytes) 2024-09-14T06:59:38.358 INFO:teuthology.orchestra.run.smithi007.stdout:371 read 0x19b5150 thru 0x19b8f51 (0x3e02 bytes) 2024-09-14T06:59:38.359 INFO:teuthology.orchestra.run.smithi007.stdout:372 read 0x836a009 thru 0x8372f45 (0x8f3d bytes) 2024-09-14T06:59:38.359 INFO:teuthology.orchestra.run.smithi007.stdout:374 punch from 0xa898bc to 0xa8abe6, (0x132a bytes) 2024-09-14T06:59:38.359 INFO:teuthology.orchestra.run.smithi007.stdout:375 writesame 0x740d34c thru 0x7419986 data_size 0x1(0xc63b bytes) 2024-09-14T06:59:38.361 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:38.360+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:38.366 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:38.366 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:38.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:59:38.380 INFO:teuthology.orchestra.run.smithi007.stdout:376 punch from 0x57cbc5e to 0x57d1059, (0x53fb bytes) 2024-09-14T06:59:38.380 INFO:teuthology.orchestra.run.smithi007.stdout:379 read 0xb6389e4 thru 0xb63dc45 (0x5262 bytes) 2024-09-14T06:59:38.389 INFO:teuthology.orchestra.run.smithi007.stdout:380 trunc from 0xe6db581 to 0x6b2ac4f 2024-09-14T06:59:38.400 INFO:teuthology.orchestra.run.smithi007.stdout:381 read 0x4209ac4 thru 0x420dbf5 (0x4132 bytes) 2024-09-14T06:59:38.411 INFO:teuthology.orchestra.run.smithi007.stdout:382 write 0xac106d8 thru 0xac1b83e (0xb167 bytes) 2024-09-14T06:59:38.419 INFO:teuthology.orchestra.run.smithi007.stdout:383 write 0xbf4a298 thru 0xbf4b766 (0x14cf bytes) 2024-09-14T06:59:38.420 INFO:teuthology.orchestra.run.smithi007.stdout:385 write 0x558ae1c thru 0x558d302 (0x24e7 bytes) 2024-09-14T06:59:38.422 INFO:teuthology.orchestra.run.smithi007.stdout:386 write 0x74a791d thru 0x74ab8f7 (0x3fdb bytes) 2024-09-14T06:59:38.424 INFO:teuthology.orchestra.run.smithi007.stdout:387 writesame 0xb27bbc4 thru 0xb284988 data_size 0x1(0x8dc5 bytes) 2024-09-14T06:59:38.440 INFO:teuthology.orchestra.run.smithi007.stdout:388 read 0xb404c8c thru 0xb413a63 (0xedd8 bytes) 2024-09-14T06:59:38.440 INFO:teuthology.orchestra.run.smithi007.stdout:390 read 0x8c9a6a3 thru 0x8ca4c72 (0xa5d0 bytes) 2024-09-14T06:59:38.441 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.441 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.448 INFO:teuthology.orchestra.run.smithi007.stdout:394 writesame 0xd09a8bf thru 0xd0a7536 data_size 0x24(0xcc78 bytes) 2024-09-14T06:59:38.449 INFO:teuthology.orchestra.run.smithi007.stdout:395 read 0x3c4deee thru 0x3c5a47f (0xc592 bytes) 2024-09-14T06:59:38.450 INFO:teuthology.orchestra.run.smithi007.stdout:397 read 0x82bd9ad thru 0x82bf5c4 (0x1c18 bytes) 2024-09-14T06:59:38.452 INFO:teuthology.orchestra.run.smithi007.stdout:398 trunc from 0xd0a7537 to 0xb50f0ec 2024-09-14T06:59:38.462 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:38.461+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:38.464 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.464 INFO:teuthology.orchestra.run.smithi007.stdout:401 read 0x42ebc0b thru 0x42ec24c (0x642 bytes) 2024-09-14T06:59:38.468 INFO:teuthology.orchestra.run.smithi007.stdout:402 trunc from 0xb50f0ec to 0xebc5fca 2024-09-14T06:59:38.474 INFO:teuthology.orchestra.run.smithi007.stdout:403 write 0x6326e23 thru 0x632f54a (0x8728 bytes) 2024-09-14T06:59:38.476 INFO:teuthology.orchestra.run.smithi007.stdout:404 writesame 0xa31c27d thru 0xa31e4f5 data_size 0x5(0x2279 bytes) 2024-09-14T06:59:38.478 INFO:teuthology.orchestra.run.smithi007.stdout:405 writesame 0x3e75cc2 thru 0x3e7f78f data_size 0xf(0x9ace bytes) 2024-09-14T06:59:38.480 INFO:teuthology.orchestra.run.smithi007.stdout:406 read 0x7320084 thru 0x7328aa3 (0x8a20 bytes) 2024-09-14T06:59:38.481 INFO:teuthology.orchestra.run.smithi007.stdout:407 read 0x3e3f39 thru 0x3f0b60 (0xcc28 bytes) 2024-09-14T06:59:38.494 INFO:teuthology.orchestra.run.smithi007.stdout:408 trunc from 0xebc5fca to 0x3691e97 2024-09-14T06:59:38.504 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:38.504 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:38.515 INFO:teuthology.orchestra.run.smithi007.stdout:410 writesame 0x9a173bf thru 0x9a1f5b0 data_size 0x1(0x81f2 bytes) 2024-09-14T06:59:38.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:59:38.517 INFO:teuthology.orchestra.run.smithi007.stdout:411 read 0x86c705f thru 0x86cb60a (0x45ac bytes) 2024-09-14T06:59:38.517 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.517 INFO:teuthology.orchestra.run.smithi007.stdout:413 read 0x4713abe thru 0x471a4fe (0x6a41 bytes) 2024-09-14T06:59:38.517 INFO:teuthology.orchestra.run.smithi007.stdout:414 read 0x15b91d5 thru 0x15be7a1 (0x55cd bytes) 2024-09-14T06:59:38.519 INFO:teuthology.orchestra.run.smithi007.stdout:415 write 0x6e6ff22 thru 0x6e7d94f (0xda2e bytes) 2024-09-14T06:59:38.519 INFO:teuthology.orchestra.run.smithi007.stdout:416 writesame 0x3b94257 thru 0x3b9c845 data_size 0xb(0x85ef bytes) 2024-09-14T06:59:38.522 INFO:teuthology.orchestra.run.smithi007.stdout:417 punch from 0x2dff6ce to 0x2e0c047, (0xc979 bytes) 2024-09-14T06:59:38.522 INFO:teuthology.orchestra.run.smithi007.stdout:418 read 0x7d0f981 thru 0x7d1cae3 (0xd163 bytes) 2024-09-14T06:59:38.522 INFO:teuthology.orchestra.run.smithi007.stdout:419 writesame 0xb05ab6 thru 0xb0a59b data_size 0x1(0x4ae6 bytes) 2024-09-14T06:59:38.531 INFO:teuthology.orchestra.run.smithi007.stdout:420 read 0x26cc7a thru 0x2797a9 (0xcb30 bytes) 2024-09-14T06:59:38.533 INFO:teuthology.orchestra.run.smithi007.stdout:423 read 0x2b52d41 thru 0x2b614ef (0xe7af bytes) 2024-09-14T06:59:38.533 INFO:teuthology.orchestra.run.smithi007.stdout:426 read 0x76674d thru 0x7692a4 (0x2b58 bytes) 2024-09-14T06:59:38.534 INFO:teuthology.orchestra.run.smithi007.stdout:427 writesame 0x3550db5 thru 0x3556e30 data_size 0x64(0x607c bytes) 2024-09-14T06:59:38.535 INFO:teuthology.orchestra.run.smithi007.stdout:428 read 0x28fd437 thru 0x2901372 (0x3f3c bytes) 2024-09-14T06:59:38.545 INFO:teuthology.orchestra.run.smithi007.stdout:430 write 0xd82c686 thru 0xd82e4ae (0x1e29 bytes) 2024-09-14T06:59:38.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:59:38.557 INFO:teuthology.orchestra.run.smithi007.stdout:431 trunc from 0xd82e4af to 0x40b708b 2024-09-14T06:59:38.563 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:38.562+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:38.572 INFO:teuthology.orchestra.run.smithi007.stdout:432 trunc from 0x40b708b to 0x9cb581b 2024-09-14T06:59:38.577 INFO:teuthology.orchestra.run.smithi007.stdout:433 punch from 0x583fb1c to 0x5849f66, (0xa44a bytes) 2024-09-14T06:59:38.577 INFO:teuthology.orchestra.run.smithi007.stdout:434 write 0x58f7be5 thru 0x58f9ea5 (0x22c1 bytes) 2024-09-14T06:59:38.579 INFO:teuthology.orchestra.run.smithi007.stdout:435 writesame 0x65c254f thru 0x65c2f48 data_size 0x1(0x9fa bytes) 2024-09-14T06:59:38.581 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.581 INFO:teuthology.orchestra.run.smithi007.stdout:437 writesame 0x7102148 thru 0x71021cc data_size 0x1(0x85 bytes) 2024-09-14T06:59:38.582 INFO:teuthology.orchestra.run.smithi007.stdout:438 write 0x358930e thru 0x358f5f7 (0x62ea bytes) 2024-09-14T06:59:38.585 INFO:teuthology.orchestra.run.smithi007.stdout:439 trunc from 0x9cb581b to 0x8f165a5 2024-09-14T06:59:38.596 INFO:teuthology.orchestra.run.smithi007.stdout:440 trunc from 0x8f165a5 to 0xabd2bd5 2024-09-14T06:59:38.601 INFO:teuthology.orchestra.run.smithi007.stdout:442 writesame 0x59f2a23 thru 0x59fb351 data_size 0x1d(0x892f bytes) 2024-09-14T06:59:38.609 INFO:teuthology.orchestra.run.smithi007.stdout:443 write 0xb6324f1 thru 0xb63328e (0xd9e bytes) 2024-09-14T06:59:38.611 INFO:teuthology.orchestra.run.smithi007.stdout:444 writesame 0x1dfd29f thru 0x1dfda25 data_size 0x1(0x787 bytes) 2024-09-14T06:59:38.614 INFO:teuthology.orchestra.run.smithi007.stdout:445 trunc from 0xb63328f to 0xcb6fde0 2024-09-14T06:59:38.619 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.619 INFO:teuthology.orchestra.run.smithi007.stdout:448 read 0x8132f3a thru 0x8138a68 (0x5b2f bytes) 2024-09-14T06:59:38.620 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.620 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.620 INFO:teuthology.orchestra.run.smithi007.stdout:451 punch from 0xb28dc59 to 0xb292f35, (0x52dc bytes) 2024-09-14T06:59:38.620 INFO:teuthology.orchestra.run.smithi007.stdout:452 punch from 0x7b4ac07 to 0x7b52406, (0x77ff bytes) 2024-09-14T06:59:38.620 INFO:teuthology.orchestra.run.smithi007.stdout:453 read 0x40c5d0f thru 0x40d4b14 (0xee06 bytes) 2024-09-14T06:59:38.620 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.620 INFO:teuthology.orchestra.run.smithi007.stdout:456 read 0x7c48123 thru 0x7c48684 (0x562 bytes) 2024-09-14T06:59:38.620 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.620 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.621 INFO:teuthology.orchestra.run.smithi007.stdout:461 writesame 0x46becbe thru 0x46c2d62 data_size 0x13(0x40a5 bytes) 2024-09-14T06:59:38.622 INFO:teuthology.orchestra.run.smithi007.stdout:463 write 0x6a1ec57 thru 0x6a1f630 (0x9da bytes) 2024-09-14T06:59:38.623 INFO:teuthology.orchestra.run.smithi007.stdout:464 read 0x344369e thru 0x344eec2 (0xb825 bytes) 2024-09-14T06:59:38.623 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.624 INFO:teuthology.orchestra.run.smithi007.stdout:467 read 0x344b237 thru 0x344cd5f (0x1b29 bytes) 2024-09-14T06:59:38.624 INFO:teuthology.orchestra.run.smithi007.stdout:468 writesame 0x653f29 thru 0x655119 data_size 0x1(0x11f1 bytes) 2024-09-14T06:59:38.626 INFO:teuthology.orchestra.run.smithi007.stdout:469 punch from 0xa004cc9 to 0xa009f72, (0x52a9 bytes) 2024-09-14T06:59:38.627 INFO:teuthology.orchestra.run.smithi007.stdout:470 read 0x6b67a9e thru 0x6b67ef0 (0x453 bytes) 2024-09-14T06:59:38.627 INFO:teuthology.orchestra.run.smithi007.stdout:471 writesame 0x8a67348 thru 0x8a76426 data_size 0x17(0xf0df bytes) 2024-09-14T06:59:38.639 INFO:teuthology.orchestra.run.smithi007.stdout:472 trunc from 0xcb6fde0 to 0x4212c56 2024-09-14T06:59:38.642 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:38.642 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:38.648 INFO:teuthology.orchestra.run.smithi007.stdout:474 write 0x2f8467e thru 0x2f8d2a6 (0x8c29 bytes) 2024-09-14T06:59:38.650 INFO:teuthology.orchestra.run.smithi007.stdout:475 read 0x1b7e0b6 thru 0x1b881ea (0xa135 bytes) 2024-09-14T06:59:38.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:59:38.659 INFO:teuthology.orchestra.run.smithi007.stdout:476 write 0x69ec54a thru 0x69f8c67 (0xc71e bytes) 2024-09-14T06:59:38.661 INFO:teuthology.orchestra.run.smithi007.stdout:478 writesame 0x34b3967 thru 0x34b5ca3 data_size 0x3(0x233d bytes) 2024-09-14T06:59:38.664 INFO:teuthology.orchestra.run.smithi007.stdout:479 trunc from 0x69f8c68 to 0x7bb3a5b 2024-09-14T06:59:38.664 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:38.663+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:38.683 INFO:teuthology.orchestra.run.smithi007.stdout:480 write 0xdfa49f4 thru 0xdfa7639 (0x2c46 bytes) 2024-09-14T06:59:38.684 INFO:teuthology.orchestra.run.smithi007.stdout:481 write 0x22983b7 thru 0x229f1ab (0x6df5 bytes) 2024-09-14T06:59:38.686 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.686 INFO:teuthology.orchestra.run.smithi007.stdout:484 read 0x3720f45 thru 0x372ef74 (0xe030 bytes) 2024-09-14T06:59:38.687 INFO:teuthology.orchestra.run.smithi007.stdout:485 read 0x58605f1 thru 0x5866ebd (0x68cd bytes) 2024-09-14T06:59:38.687 INFO:teuthology.orchestra.run.smithi007.stdout:486 writesame 0x91b1d28 thru 0x91c1c3d data_size 0x67(0xff16 bytes) 2024-09-14T06:59:38.689 INFO:teuthology.orchestra.run.smithi007.stdout:488 write 0xa04f9c5 thru 0xa05578b (0x5dc7 bytes) 2024-09-14T06:59:38.691 INFO:teuthology.orchestra.run.smithi007.stdout:489 punch from 0x663d950 to 0x6647ec2, (0xa572 bytes) 2024-09-14T06:59:38.691 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.691 INFO:teuthology.orchestra.run.smithi007.stdout:492 read 0xd6865f5 thru 0xd693316 (0xcd22 bytes) 2024-09-14T06:59:38.691 INFO:teuthology.orchestra.run.smithi007.stdout:493 writesame 0x5fccf2a thru 0x5fcf341 data_size 0x3(0x2418 bytes) 2024-09-14T06:59:38.693 INFO:teuthology.orchestra.run.smithi007.stdout:494 write 0x1bfddfe thru 0x1c03276 (0x5479 bytes) 2024-09-14T06:59:38.695 INFO:teuthology.orchestra.run.smithi007.stdout:495 writesame 0xc924bcd thru 0xc92575e data_size 0x1(0xb92 bytes) 2024-09-14T06:59:38.700 INFO:teuthology.orchestra.run.smithi007.stdout:496 trunc from 0xdfa763a to 0xaef639c 2024-09-14T06:59:38.706 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:38.717 INFO:teuthology.orchestra.run.smithi007.stdout:497 write 0xbe27687 thru 0xbe324e7 (0xae61 bytes) 2024-09-14T06:59:38.719 INFO:teuthology.orchestra.run.smithi007.stdout:498 write 0x737bdbd thru 0x738670a (0xa94e bytes) 2024-09-14T06:59:38.720 INFO:teuthology.orchestra.run.smithi007.stdout:500 write 0x3684251 thru 0x368b6ca (0x747a bytes) 2024-09-14T06:59:38.728 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:38.728 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:38.731 INFO:teuthology.orchestra.run.smithi007.stdout:501 write 0xe8a7ad8 thru 0xe8b79ef (0xff18 bytes) 2024-09-14T06:59:38.733 INFO:teuthology.orchestra.run.smithi007.stdout:503 read 0xb4cb92f thru 0xb4ccb8c (0x125e bytes) 2024-09-14T06:59:38.733 INFO:teuthology.orchestra.run.smithi007.stdout:504 read 0x81720f6 thru 0x817a64c (0x8557 bytes) 2024-09-14T06:59:38.734 INFO:teuthology.orchestra.run.smithi007.stdout:508 read 0x2ed9448 thru 0x2ee6a55 (0xd60e bytes) 2024-09-14T06:59:38.735 INFO:teuthology.orchestra.run.smithi007.stdout:509 trunc from 0xe8b79f0 to 0xece58ca 2024-09-14T06:59:38.740 INFO:teuthology.orchestra.run.smithi007.stdout:510 read 0x2fd4bb0 thru 0x2fe1819 (0xcc6a bytes) 2024-09-14T06:59:38.740 INFO:teuthology.orchestra.run.smithi007.stdout:512 read 0x5386736 thru 0x538bb43 (0x540e bytes) 2024-09-14T06:59:38.741 INFO:teuthology.orchestra.run.smithi007.stdout:513 read 0x6802253 thru 0x68102ed (0xe09b bytes) 2024-09-14T06:59:38.747 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:38.747 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:38.748 INFO:teuthology.orchestra.run.smithi007.stdout:514 trunc from 0xece58ca to 0x923bb04 2024-09-14T06:59:38.756 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.765 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:38.765+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:38.767 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:38.767 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:38.769 INFO:teuthology.orchestra.run.smithi007.stdout:516 writesame 0xdae4947 thru 0xdae584a data_size 0x4(0xf04 bytes) 2024-09-14T06:59:38.770 INFO:teuthology.orchestra.run.smithi007.stdout:518 write 0x13a320e thru 0x13a844d (0x5240 bytes) 2024-09-14T06:59:38.772 INFO:teuthology.orchestra.run.smithi007.stdout:521 writesame 0xaa43a5d thru 0xaa52c69 data_size 0x55(0xf20d bytes) 2024-09-14T06:59:38.774 INFO:teuthology.orchestra.run.smithi007.stdout:522 read 0xbf02050 thru 0xbf047e7 (0x2798 bytes) 2024-09-14T06:59:38.774 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.774 INFO:teuthology.orchestra.run.smithi007.stdout:525 punch from 0x344a970 to 0x3457029, (0xc6b9 bytes) 2024-09-14T06:59:38.774 INFO:teuthology.orchestra.run.smithi007.stdout:527 write 0xa912e20 thru 0xa91db64 (0xad45 bytes) 2024-09-14T06:59:38.776 INFO:teuthology.orchestra.run.smithi007.stdout:528 write 0x847565f thru 0x8480fdd (0xb97f bytes) 2024-09-14T06:59:38.778 INFO:teuthology.orchestra.run.smithi007.stdout:529 write 0x37dbf72 thru 0x37e7471 (0xb500 bytes) 2024-09-14T06:59:38.781 INFO:teuthology.orchestra.run.smithi007.stdout:530 trunc from 0xdae584b to 0xc0984a9 2024-09-14T06:59:38.788 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:38.788 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:38.789 INFO:teuthology.orchestra.run.smithi007.stdout:531 write 0x7b1f4c6 thru 0x7b23870 (0x43ab bytes) 2024-09-14T06:59:38.791 INFO:teuthology.orchestra.run.smithi007.stdout:532 write 0x549ca72 thru 0x549e9f1 (0x1f80 bytes) 2024-09-14T06:59:38.799 INFO:teuthology.orchestra.run.smithi007.stdout:535 writesame 0xc8ad83e thru 0xc8b4bc4 data_size 0x5b(0x7387 bytes) 2024-09-14T06:59:38.799 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1838 3.14 3:2f4a3dac:::rbd_data.114fe955cd3f.0000000000000000:head [stat out=16b,zero 21682238~29575] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.799327+0000", 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.000323407, 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035063599999999998, 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:38.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1838 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.799327+0000", 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.799327+0000", 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:38.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.799328+0000", 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.799331+0000", 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5530000000000001e-06 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.799341+0000", 2024-09-14T06:59:38.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7100000000000002e-06 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.799348+0000", 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.922e-06 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.799392+0000", 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3714999999999997e-05 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:38.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.799441+0000", 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:38.804 INFO:teuthology.orchestra.run.smithi007.stdout:536 writesame 0x90bc6a0 thru 0x90cbc10 data_size 0x1(0xf571 bytes) 2024-09-14T06:59:38.806 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:38.809 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:59:38.824 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:38.824 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:38.834 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:59:38.834 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:59:38.835 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T06:59:38.835 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-09-14T06:59:38.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0 2024-09-14T06:59:38.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:59:38.836 INFO:teuthology.orchestra.run.smithi007.stdout:537 trunc from 0xc8b4bc5 to 0x699a526 2024-09-14T06:59:38.846 INFO:teuthology.orchestra.run.smithi007.stdout:540 punch from 0x43b026f to 0x43b2c78, (0x2a09 bytes) 2024-09-14T06:59:38.847 INFO:teuthology.orchestra.run.smithi007.stdout:541 write 0x20f4213 thru 0x20f5b59 (0x1947 bytes) 2024-09-14T06:59:38.847 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:38.847 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:38.848 INFO:teuthology.orchestra.run.smithi007.stdout:542 writesame 0x4e0e420 thru 0x4e12a47 data_size 0x8(0x4628 bytes) 2024-09-14T06:59:38.851 INFO:teuthology.orchestra.run.smithi007.stdout:543 read 0x3e0106c thru 0x3e0f0fc (0xe091 bytes) 2024-09-14T06:59:38.856 INFO:teuthology.orchestra.run.smithi007.stdout:545 trunc from 0x699a526 to 0x99dc020 2024-09-14T06:59:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:546 punch from 0x6728559 to 0x672c4c6, (0x3f6d bytes) 2024-09-14T06:59:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:547 read 0x6d557d0 thru 0x6d56b61 (0x1392 bytes) 2024-09-14T06:59:38.862 INFO:teuthology.orchestra.run.smithi007.stdout:548 punch from 0x66ee0e8 to 0x66fb07c, (0xcf94 bytes) 2024-09-14T06:59:38.863 INFO:teuthology.orchestra.run.smithi007.stdout:549 trunc from 0x99dc020 to 0x90176e5 2024-09-14T06:59:38.866 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:38.865+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:38.873 INFO:teuthology.orchestra.run.smithi007.stdout:551 writesame 0x57c93bf thru 0x57d847d data_size 0x1(0xf0bf bytes) 2024-09-14T06:59:38.900 INFO:teuthology.orchestra.run.smithi007.stdout:552 write 0x45e0565 thru 0x45e103c (0xad8 bytes) 2024-09-14T06:59:38.902 INFO:teuthology.orchestra.run.smithi007.stdout:553 writesame 0x50e1a4d thru 0x50e2e1e data_size 0x2(0x13d2 bytes) 2024-09-14T06:59:38.914 INFO:teuthology.orchestra.run.smithi007.stdout:554 write 0xc1509af thru 0xc15e98c (0xdfde bytes) 2024-09-14T06:59:38.922 INFO:teuthology.orchestra.run.smithi007.stdout:555 writesame 0xc658c9a thru 0xc667f4b data_size 0x6(0xf2b2 bytes) 2024-09-14T06:59:38.923 INFO:teuthology.orchestra.run.smithi007.stdout:556 write 0x15a3760 thru 0x15b19ee (0xe28f bytes) 2024-09-14T06:59:38.933 INFO:teuthology.orchestra.run.smithi007.stdout:557 writesame 0xd06dd25 thru 0xd07d476 data_size 0x1(0xf752 bytes) 2024-09-14T06:59:38.934 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.935 INFO:teuthology.orchestra.run.smithi007.stdout:561 trunc from 0xd07d477 to 0xe09315e 2024-09-14T06:59:38.942 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.942 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.942 INFO:teuthology.orchestra.run.smithi007.stdout:565 write 0x2548a39 thru 0x254fb1c (0x70e4 bytes) 2024-09-14T06:59:38.944 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.944 INFO:teuthology.orchestra.run.smithi007.stdout:567 write 0xa7c602b thru 0xa7ce2bd (0x8293 bytes) 2024-09-14T06:59:38.945 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:38.945 INFO:teuthology.orchestra.run.smithi007.stdout:569 clone 3 order 19 su 65536 sc 9 2024-09-14T06:59:38.955 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:38.955 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:38.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:59:38.967 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:38.966+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:39.069 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:39.068+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.359252084000001, 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T06:59:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T06:59:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T06:59:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:20.763463+0000", 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349587666000001, 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045167810000000001, 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 5 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763463+0000", 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763463+0000", 2024-09-14T06:59:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763465+0000", 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1679999999999998e-06 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763476+0000", 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0883000000000001e-05 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763494+0000", 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8233999999999999e-05 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763503+0000", 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0489999999999993e-06 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763572+0000", 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8830000000000003e-05 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763704+0000", 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132399 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.763821+0000", 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011696000000000001 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.764623+0000", 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080120500000000004 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767909+0000", 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003285966 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767947+0000", 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8269999999999998e-05 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.767980+0000", 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2817000000000001e-05 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:12 3.9 3:91fcf338:::rbd_header.114f4f9a926b:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:30.812434+0000", 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.3006167430000009, 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018126419999999999, 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812434+0000", 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812434+0000", 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812435+0000", 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812441+0000", 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2060000000000004e-06 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812451+0000", 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5689999999999994e-06 2024-09-14T06:59:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812458+0000", 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8349999999999998e-06 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812525+0000", 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6879999999999997e-05 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812708+0000", 2024-09-14T06:59:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018373599999999999 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.812981+0000", 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000272434 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.813665+0000", 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068414099999999998 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814178+0000", 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051320000000000001 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814215+0000", 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6949999999999997e-05 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:30.814246+0000", 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1309999999999997e-05 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1808 3.14 3:2f4a3dac:::rbd_data.114fe955cd3f.0000000000000000:head [stat,zero 20971520~2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.782216+0000", 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.33083425999999999, 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002288507, 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1808 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782216+0000", 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782216+0000", 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782219+0000", 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5170000000000002e-06 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782236+0000", 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7312e-05 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782259+0000", 2024-09-14T06:59:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2988000000000001e-05 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782272+0000", 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.238e-05 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782346+0000", 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4099999999999999e-05 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782472+0000", 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000125928 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.784505+0000", 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002033282 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1809 3.13 3:ca375c0d:::rbd_data.114fe955cd3f.0000000000000001:head [stat,zero 20971520~2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.782297+0000", 2024-09-14T06:59:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.330754033, 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002311727, 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1809 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782297+0000", 2024-09-14T06:59:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782297+0000", 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782297+0000", 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3600000000000004e-07 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782300+0000", 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6900000000000001e-06 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782308+0000", 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5909999999999996e-06 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782314+0000", 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6790000000000002e-06 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782381+0000", 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7146999999999999e-05 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782470+0000", 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9192999999999998e-05 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.784608+0000", 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021378909999999998 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1816 3.13 3:c8476c59:::rbd_data.114fe955cd3f.0000000000000008:head [stat,zero 19922944~3037259] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.782342+0000", 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.33070844599999999, 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024039999999999999, 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1816 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782342+0000", 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782342+0000", 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782343+0000", 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9699999999999996e-07 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782346+0000", 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9900000000000002e-06 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782354+0000", 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9910000000000002e-06 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782360+0000", 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0820000000000004e-06 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782867+0000", 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050716300000000004 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.782958+0000", 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1278000000000003e-05 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.784746+0000", 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017878989999999999 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1830 3.9 3:93377bc8:::rbd_header.114fe955cd3f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.795173+0000", 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.317877519, 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016937320000000001, 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1830 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795173+0000", 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795173+0000", 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795175+0000", 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5820000000000001e-06 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795182+0000", 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7090000000000002e-06 2024-09-14T06:59:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795199+0000", 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6569999999999999e-05 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795208+0000", 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5140000000000007e-06 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795273+0000", 2024-09-14T06:59:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4457999999999998e-05 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795331+0000", 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8056000000000001e-05 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795462+0000", 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013099699999999999 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.795924+0000", 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046226999999999998 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.796803+0000", 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087842699999999998 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.796840+0000", 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7156999999999997e-05 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.796867+0000", 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6991999999999998e-05 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1840 3.14 3:2f4a3dac:::rbd_data.114fe955cd3f.0000000000000000:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.837101+0000", 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.27594924799999998, 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003167717, 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1840 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837101+0000", 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837101+0000", 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837104+0000", 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0630000000000002e-06 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837118+0000", 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3974999999999999e-05 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837146+0000", 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7820000000000001e-05 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837160+0000", 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4233000000000001e-05 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837267+0000", 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010677499999999999 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837375+0000", 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107552 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837508+0000", 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013287999999999999 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838765+0000", 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012571430000000001 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.840123+0000", 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013579449999999999 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.840178+0000", 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5451999999999997e-05 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.840269+0000", 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-14T06:59:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1841 3.13 3:ca375c0d:::rbd_data.114fe955cd3f.0000000000000001:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.837200+0000", 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.27585106599999998, 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023519750000000001, 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1841 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837200+0000", 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837200+0000", 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837200+0000", 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3099999999999998e-07 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837203+0000", 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4470000000000002e-06 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837213+0000", 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6220000000000004e-06 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837239+0000", 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6227000000000001e-05 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837313+0000", 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4092999999999997e-05 2024-09-14T06:59:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837392+0000", 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8403000000000001e-05 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837497+0000", 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010529199999999999 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838780+0000", 2024-09-14T06:59:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012831780000000001 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.839407+0000", 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062700400000000002 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.839491+0000", 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3364999999999999e-05 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.839551+0000", 2024-09-14T06:59:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0813000000000002e-05 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1848 3.13 3:c8476c59:::rbd_data.114fe955cd3f.0000000000000008:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.27578220599999997, 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024472560000000001, 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1848 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837269+0000", 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2399999999999998e-07 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837272+0000", 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0510000000000001e-06 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837340+0000", 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7898000000000006e-05 2024-09-14T06:59:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.837347+0000", 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0700000000000001e-06 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838012+0000", 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066460300000000005 2024-09-14T06:59:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838104+0000", 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2682999999999993e-05 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.839716+0000", 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016113270000000001 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4431.0:1843 3.12 e30/24 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head v 30'61, mlcod=30'59)", 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.838130+0000", 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.27492019000000001, 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022974200000000001, 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838130+0000", 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838130+0000", 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838132+0000", 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2669999999999999e-06 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838138+0000", 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8880000000000002e-06 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838150+0000", 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.208e-05 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838157+0000", 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9399999999999996e-06 2024-09-14T06:59:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838268+0000", 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000111321 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.839145+0000", 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087674499999999995 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.840392+0000", 2024-09-14T06:59:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001247083 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.840428+0000", 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6096000000000001e-05 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "pg_update_log_missing(3.12 epoch 30/24 rep_tid 61 entries 30'62 (0'0) error 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head by client.4431.0:1846 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'59)", 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.838543+0000", 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.27450772000000001, 2024-09-14T06:59:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021820009999999998, 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838543+0000", 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838543+0000", 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838544+0000", 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5300000000000004e-07 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838551+0000", 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4290000000000004e-06 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838566+0000", 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.524e-05 2024-09-14T06:59:39.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.838579+0000", 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.325e-05 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.839775+0000", 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011957440000000001 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.840725+0000", 2024-09-14T06:59:39.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094968500000000003 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1860 3.13 3:c8476c59:::rbd_data.114fe955cd3f.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.849038+0000", 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.26401245299999998, 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002524732, 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1860 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.849038+0000", 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.849038+0000", 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.849040+0000", 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.762e-06 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.849054+0000", 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.412e-05 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.849068+0000", 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4456e-05 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.849083+0000", 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4062e-05 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.849160+0000", 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7767000000000003e-05 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.849244+0000", 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3251999999999995e-05 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.849474+0000", 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023040499999999999 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.850854+0000", 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013803960000000001 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.851444+0000", 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000589812 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.851515+0000", 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0928999999999998e-05 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.851563+0000", 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7771000000000001e-05 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.183 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1872 3.14 3:2f4a3dac:::rbd_data.114fe955cd3f.0000000000000000:head [stat,zero 15728640~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.863701+0000", 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.24934985100000001, 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031557030000000002, 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1872 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863701+0000", 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863701+0000", 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863703+0000", 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5979999999999999e-06 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863735+0000", 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1575000000000002e-05 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863763+0000", 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8438000000000001e-05 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863778+0000", 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4423999999999999e-05 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864168+0000", 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039043699999999999 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864334+0000", 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016599599999999999 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.866856+0000", 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025222349999999998 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.187 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1873 3.13 3:ca375c0d:::rbd_data.114fe955cd3f.0000000000000001:head [stat,zero 15728640~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.863812+0000", 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.24923884900000001, 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025556289999999998, 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1873 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863812+0000", 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863812+0000", 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863812+0000", 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5799999999999999e-07 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863815+0000", 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0019999999999998e-06 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863825+0000", 2024-09-14T06:59:39.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2089999999999994e-06 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863845+0000", 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0485000000000001e-05 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863906+0000", 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0798000000000003e-05 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863992+0000", 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6341000000000006e-05 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.866367+0000", 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023751359999999999 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1880 3.13 3:c8476c59:::rbd_data.114fe955cd3f.0000000000000008:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.863866+0000", 2024-09-14T06:59:39.192 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.24918423100000001, 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030014170000000001, 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1880 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863866+0000", 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863866+0000", 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863867+0000", 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863870+0000", 2024-09-14T06:59:39.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.717e-06 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863877+0000", 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.79e-06 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.863885+0000", 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7910000000000005e-06 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864293+0000", 2024-09-14T06:59:39.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00040738599999999998 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864670+0000", 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00037738399999999998 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.866868+0000", 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021977989999999998 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "pg_update_log_missing(3.12 epoch 30/24 rep_tid 62 entries 30'63 (0'0) error 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head by client.4431.0:1875 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'61)", 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.864391+0000", 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.24865965100000001, 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023575390000000001, 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864391+0000", 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864391+0000", 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864392+0000", 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864400+0000", 2024-09-14T06:59:39.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2889999999999998e-06 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864418+0000", 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7504999999999999e-05 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864431+0000", 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2955e-05 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864523+0000", 2024-09-14T06:59:39.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2108999999999995e-05 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.866748+0000", 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022254589999999999 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "pg_update_log_missing(3.12 epoch 30/24 rep_tid 63 entries 30'64 (0'0) error 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head by client.4431.0:1878 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'61)", 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.864803+0000", 2024-09-14T06:59:39.200 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.24824775900000001, 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019666620000000001, 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864803+0000", 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864803+0000", 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864804+0000", 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2500000000000004e-07 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864810+0000", 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5359999999999998e-06 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864825+0000", 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4398e-05 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864879+0000", 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4441999999999998e-05 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.864964+0000", 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4517000000000004e-05 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.866769+0000", 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018059440000000001 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1893 3.14 3:2f4a3dac:::rbd_data.114fe955cd3f.0000000000000000:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.902924+0000", 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.210126375, 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020016349999999999, 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1893 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.902924+0000", 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.902924+0000", 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.902927+0000", 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7609999999999999e-06 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.902937+0000", 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6819999999999998e-06 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.902959+0000", 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1999000000000001e-05 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.902969+0000", 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0492000000000001e-05 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.903042+0000", 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2616999999999999e-05 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.903103+0000", 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-05 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.903380+0000", 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027690700000000001 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.904560+0000", 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001179851 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.904862+0000", 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030219900000000001 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.904895+0000", 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2283999999999998e-05 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.904926+0000", 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1282999999999998e-05 2024-09-14T06:59:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:1914 3.13 3:ca375c0d:::rbd_data.114fe955cd3f.0000000000000001:head [stat,write 2766688~57999] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:38.924181+0000", 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.18886930099999999, 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001739946, 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1914 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.924181+0000", 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.924181+0000", 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.924182+0000", 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.924220+0000", 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7410000000000003e-05 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.924231+0000", 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1608e-05 2024-09-14T06:59:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.924239+0000", 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4399999999999999e-06 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.924302+0000", 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3136e-05 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.924401+0000", 2024-09-14T06:59:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9053000000000002e-05 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.924578+0000", 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000177283 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.925746+0000", 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001168337 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.925801+0000", 2024-09-14T06:59:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4611000000000003e-05 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.925866+0000", 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4530999999999996e-05 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:38.925921+0000", 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5587999999999997e-05 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:39.216 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:39.217 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:59:39.218 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:39.216+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:39.274 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.6 primary-affinity to 0 (802) 2024-09-14T06:59:39.277 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T06:59:39.325 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:39.325 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:39.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.326 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2002 3.6 3:61705fc6:::rbd_header.114fe2819363:head [call rbd.set_stripe_unit_count in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:39.326 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:39.326 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.0016371070000000001, 2024-09-14T06:59:39.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001654708, 2024-09-14T06:59:39.327 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.327 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.327 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:59:39.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.327 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2002 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6899999999999997e-07 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323802+0000", 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9641999999999999e-05 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323813+0000", 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1263999999999999e-05 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323818+0000", 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5390000000000001e-06 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323858+0000", 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0556000000000003e-05 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323904+0000", 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5383000000000002e-05 2024-09-14T06:59:39.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.324099+0000", 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000195157 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 1 2024-09-14T06:59:39.331 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:39.335 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:59:39.336 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:39.335+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:39.347 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone2 from 0xe09315e (overlap 0xbe885d) to 0x3928e8 2024-09-14T06:59:39.418 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:39.417+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:39.440 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:39.440 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:39.440 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:39.440 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:39.440 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:39.449 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:59:39.519 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:39.519+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:39.594 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:39.594 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:39.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:39.594 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:39.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.594 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:25 3.9 e27/24 3:91fcf338:::rbd_header.114f4f9a926b:head v 27'2, mlcod=0'0)", 2024-09-14T06:59:39.594 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:30.823412+0000", 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 8.7684767370000003, 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071347200000000004, 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823412+0000", 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823412+0000", 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823414+0000", 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823424+0000", 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8560000000000002e-06 2024-09-14T06:59:39.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823449+0000", 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4833000000000001e-05 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823462+0000", 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3273e-05 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823591+0000", 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012913099999999999 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.823643+0000", 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.181e-05 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.824092+0000", 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00044955599999999999 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:30.824126+0000", 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.349e-05 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1828 3.19 3:9a80ba12:::rbd_data.114fe955cd3f.0000000000000004:head [stat,write 9030258~8064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.791778+0000", 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.800111613, 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001084331, 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.599 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1828 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.791778+0000", 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.791778+0000", 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.791778+0000", 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0999999999999997e-07 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.791789+0000", 2024-09-14T06:59:39.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0361e-05 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.791805+0000", 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6171000000000002e-05 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.791811+0000", 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7769999999999999e-06 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.791852+0000", 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0799e-05 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.791899+0000", 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7148999999999999e-05 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.791981+0000", 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1990999999999995e-05 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.792506+0000", 2024-09-14T06:59:39.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00052548999999999998 2024-09-14T06:59:39.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.792807+0000", 2024-09-14T06:59:39.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030058899999999998 2024-09-14T06:59:39.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.792838+0000", 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1155000000000002e-05 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.792862+0000", 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3938999999999999e-05 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.605 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:1830 3.9 e30/24 3:93377bc8:::rbd_header.114fe955cd3f:head v 30'92, mlcod=30'90)", 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.795553+0000", 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.79633618799999994, 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001133921, 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.795553+0000", 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.795553+0000", 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.795554+0000", 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8899999999999999e-07 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.795558+0000", 2024-09-14T06:59:39.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9450000000000003e-06 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.795584+0000", 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6624000000000001e-05 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.795589+0000", 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4490000000000001e-06 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.795625+0000", 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6690999999999998e-05 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.795636+0000", 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1148999999999999e-05 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.796662+0000", 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001025282 2024-09-14T06:59:39.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.796687+0000", 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5191999999999999e-05 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1839 3.19 3:9a80ba12:::rbd_data.114fe955cd3f.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.800958+0000", 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.79093148599999996, 2024-09-14T06:59:39.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.028916818, 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1839 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.800958+0000", 2024-09-14T06:59:39.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.800958+0000", 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.800960+0000", 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.934e-06 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.800972+0000", 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2659e-05 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.801000+0000", 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7399999999999999e-05 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.801013+0000", 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.365e-05 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.801153+0000", 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013930299999999999 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.801299+0000", 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014683299999999999 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.806031+0000", 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.004731458 2024-09-14T06:59:39.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829785+0000", 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.023754016999999999 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829804+0000", 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9471999999999999e-05 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829839+0000", 2024-09-14T06:59:39.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4147999999999999e-05 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829874+0000", 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5944e-05 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop_reply(client.4431.0:1839 3.19 e30/24 ondisk, result = 0)", 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.821228+0000", 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.77066095599999995, 2024-09-14T06:59:39.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0085620130000000003, 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821228+0000", 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821228+0000", 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821229+0000", 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0100000000000004e-07 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821233+0000", 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1219999999999997e-06 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821244+0000", 2024-09-14T06:59:39.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0977e-05 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821268+0000", 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4328000000000002e-05 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829771+0000", 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0085023330000000008 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829779+0000", 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.49e-06 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829790+0000", 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0961999999999999e-05 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1844 3.19 3:9a80ba12:::rbd_data.114fe955cd3f.0000000000000004:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837247+0000", 2024-09-14T06:59:39.620 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75464244700000005, 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002587275, 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1844 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837247+0000", 2024-09-14T06:59:39.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837247+0000", 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837249+0000", 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.74e-06 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837261+0000", 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1263e-05 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837293+0000", 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2363000000000002e-05 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837317+0000", 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3540000000000002e-05 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837608+0000", 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029157199999999998 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9833999999999994e-05 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837843+0000", 2024-09-14T06:59:39.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014475999999999999 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838547+0000", 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00070457199999999999 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.839772+0000", 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012246329999999999 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.839813+0000", 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1182e-05 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.839834+0000", 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0815999999999999e-05 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1849 3.1b 3:deadecef:::rbd_data.114fe955cd3f.0000000000000009:head [stat,truncate 10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.626 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837376+0000", 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75451274899999998, 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021342330000000001, 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1849 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837376+0000", 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837376+0000", 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837378+0000", 2024-09-14T06:59:39.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.483e-06 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837385+0000", 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5129999999999999e-06 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837406+0000", 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1047e-05 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837423+0000", 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6229e-05 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837611+0000", 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018841600000000001 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837713+0000", 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000101623 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837843+0000", 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013006399999999999 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838510+0000", 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00066723900000000003 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.839451+0000", 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00094114399999999999 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.839483+0000", 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1893000000000001e-05 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.839511+0000", 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7582000000000001e-05 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.632 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1876 3.19 3:9a80ba12:::rbd_data.114fe955cd3f.0000000000000004:head [stat,zero 14776037~952603] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863737+0000", 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.72815236699999997, 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0026618179999999998, 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1876 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863737+0000", 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863737+0000", 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863739+0000", 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4870000000000001e-06 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863750+0000", 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.116e-05 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863786+0000", 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5985999999999997e-05 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863819+0000", 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2409000000000001e-05 2024-09-14T06:59:39.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863976+0000", 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015743800000000001 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864109+0000", 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013315700000000001 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866399+0000", 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022891809999999999 2024-09-14T06:59:39.636 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1881 3.1b 3:deadecef:::rbd_data.114fe955cd3f.0000000000000009:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863884+0000", 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.72800509300000005, 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0017774380000000001, 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:39.637 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1881 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863884+0000", 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863884+0000", 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863886+0000", 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5099999999999999e-06 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863893+0000", 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7300000000000005e-06 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863916+0000", 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2883000000000001e-05 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863987+0000", 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0438999999999994e-05 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864097+0000", 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000110043 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864168+0000", 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0921999999999996e-05 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.865662+0000", 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0014939109999999999 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1892 3.1b 3:deadecef:::rbd_data.114fe955cd3f.0000000000000009:head [stat,write 7210341~2776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.901062+0000", 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.69082674399999999, 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016515830000000001, 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1892 2024-09-14T06:59:39.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901062+0000", 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901062+0000", 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901064+0000", 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8759999999999999e-06 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901073+0000", 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9709999999999993e-06 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901090+0000", 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6470999999999999e-05 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901098+0000", 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1939999999999998e-06 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901181+0000", 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3553000000000005e-05 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901247+0000", 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5864999999999994e-05 2024-09-14T06:59:39.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901347+0000", 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000100039 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.901931+0000", 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058342799999999998 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.902651+0000", 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071979100000000001 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.902688+0000", 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7122999999999997e-05 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.902714+0000", 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6271999999999999e-05 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.647 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:1895 3.9 e30/24 3:93377bc8:::rbd_header.114fe955cd3f:head v 30'96, mlcod=30'94)", 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.911300+0000", 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.68058917600000002, 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069895500000000002, 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911300+0000", 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911300+0000", 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911302+0000", 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.99e-06 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911313+0000", 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1017e-05 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911338+0000", 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5038999999999999e-05 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911350+0000", 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2343e-05 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911459+0000", 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108705 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911494+0000", 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4856e-05 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911969+0000", 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00047539099999999998 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.911999+0000", 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9614000000000001e-05 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:1926 3.9 e30/24 3:93377bc8:::rbd_header.114fe955cd3f:head v 30'99, mlcod=30'97)", 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.938314+0000", 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.65357489800000002, 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00075441899999999999, 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.652 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.938314+0000", 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.938314+0000", 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.938316+0000", 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.573e-06 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.938326+0000", 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0543000000000001e-05 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.938351+0000", 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4933e-05 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.938366+0000", 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.468e-05 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.938459+0000", 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3184999999999996e-05 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.938496+0000", 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6588999999999998e-05 2024-09-14T06:59:39.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.939042+0000", 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00054608499999999997 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.939069+0000", 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6831000000000001e-05 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:1939 3.9 e31/24 3:93377bc8:::rbd_header.114fe955cd3f:head v 31'100, mlcod=30'98)", 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.279581+0000", 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.31230797500000002, 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044217299999999996, 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279581+0000", 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279581+0000", 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279582+0000", 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279679+0000", 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7362999999999996e-05 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279688+0000", 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4260000000000003e-06 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279696+0000", 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6270000000000006e-06 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279831+0000", 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013447700000000001 2024-09-14T06:59:39.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279851+0000", 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0254e-05 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.283955+0000", 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041038940000000003 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.284003+0000", 2024-09-14T06:59:39.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8049e-05 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2001 3.6 3:61705fc6:::rbd_header.114fe2819363:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.322255+0000", 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.26963405400000001, 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001433475, 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.661 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2001 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.322255+0000", 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.322255+0000", 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.322256+0000", 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0900000000000001e-07 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.322260+0000", 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7809999999999999e-06 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.322270+0000", 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9550000000000004e-06 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.322276+0000", 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2129999999999996e-06 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.322321+0000", 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5309000000000003e-05 2024-09-14T06:59:39.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.322441+0000", 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000120179 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.322652+0000", 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000210917 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323180+0000", 2024-09-14T06:59:39.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00052772599999999995 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323600+0000", 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041964599999999997 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323654+0000", 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4725000000000002e-05 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323689+0000", 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4314999999999999e-05 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2002 3.6 3:61705fc6:::rbd_header.114fe2819363:head [call rbd.set_stripe_unit_count in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.268107392, 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031864329999999998, 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.667 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2002 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6899999999999997e-07 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323802+0000", 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9641999999999999e-05 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323813+0000", 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1263999999999999e-05 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323818+0000", 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5390000000000001e-06 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323858+0000", 2024-09-14T06:59:39.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0556000000000003e-05 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323904+0000", 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5383000000000002e-05 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.324099+0000", 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000195157 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.326319+0000", 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022197139999999998 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.326908+0000", 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058982900000000005 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.326942+0000", 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3275999999999998e-05 2024-09-14T06:59:39.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.326968+0000", 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6503999999999999e-05 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2016 3.6 3:61705fc6:::rbd_header.114fe2819363:head [watch watch cookie 140020028797136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:39.673 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.336104+0000", 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.25578535899999999, 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001154913, 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2016 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.336104+0000", 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.336104+0000", 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.336105+0000", 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1800000000000005e-07 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.336107+0000", 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.937e-06 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.336115+0000", 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8720000000000002e-06 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.336119+0000", 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8909999999999997e-06 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.336159+0000", 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9941999999999998e-05 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.336228+0000", 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8792000000000001e-05 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.336380+0000", 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015160400000000001 2024-09-14T06:59:39.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337039+0000", 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00065921199999999995 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337188+0000", 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014925999999999999 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337218+0000", 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9916999999999998e-05 2024-09-14T06:59:39.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337259+0000", 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0768e-05 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2017 3.6 3:61705fc6:::rbd_header.114fe2819363:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.337448+0000", 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.25444144400000002, 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024757949999999998, 2024-09-14T06:59:39.679 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2017 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337448+0000", 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337448+0000", 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337449+0000", 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999995e-07 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337459+0000", 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0712e-05 2024-09-14T06:59:39.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337472+0000", 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2267e-05 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337479+0000", 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.294e-06 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337516+0000", 2024-09-14T06:59:39.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7184999999999997e-05 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337565+0000", 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8813999999999997e-05 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337706+0000", 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000140636 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.338110+0000", 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00040478299999999998 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.339872+0000", 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001761486 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.339902+0000", 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.987e-05 2024-09-14T06:59:39.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.339924+0000", 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1803000000000001e-05 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2026 3.6 3:61705fc6:::rbd_header.114fe2819363:head [watch unwatch cookie 140020028797136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.343327+0000", 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.24856177400000001, 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0011077870000000001, 2024-09-14T06:59:39.685 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2026 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.343327+0000", 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.343327+0000", 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.343328+0000", 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.343333+0000", 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.343348+0000", 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4442e-05 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.343355+0000", 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4880000000000001e-06 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.343404+0000", 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8671000000000002e-05 2024-09-14T06:59:39.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.343480+0000", 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5904999999999999e-05 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.343588+0000", 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108813 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.344014+0000", 2024-09-14T06:59:39.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00042532300000000002 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.344316+0000", 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00030185200000000002 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.344370+0000", 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4091000000000001e-05 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.344435+0000", 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5425000000000005e-05 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2048 3.19 3:9a80ba12:::rbd_data.114fe955cd3f.0000000000000004:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.355723+0000", 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.236166033, 2024-09-14T06:59:39.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001807538, 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2048 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355723+0000", 2024-09-14T06:59:39.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355723+0000", 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355725+0000", 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3300000000000001e-06 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355733+0000", 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1149999999999996e-06 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355751+0000", 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355761+0000", 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1940000000000004e-06 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355829+0000", 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8882999999999998e-05 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355914+0000", 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4875000000000003e-05 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.356094+0000", 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017976600000000001 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357304+0000", 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012097550000000001 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357439+0000", 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000134726 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357485+0000", 2024-09-14T06:59:39.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6557999999999997e-05 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357531+0000", 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5595e-05 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2055 3.1b 3:deadecef:::rbd_data.114fe955cd3f.0000000000000009:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.361011+0000", 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.230878002, 2024-09-14T06:59:39.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002187769, 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2055 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.361011+0000", 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.361011+0000", 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.361013+0000", 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3909999999999999e-06 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.361018+0000", 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3909999999999997e-06 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.361031+0000", 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3439e-05 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.361038+0000", 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1380000000000004e-06 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.361094+0000", 2024-09-14T06:59:39.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6520000000000001e-05 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.361179+0000", 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4781e-05 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.361369+0000", 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018997299999999999 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362994+0000", 2024-09-14T06:59:39.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016250769999999999 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363096+0000", 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010206 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363150+0000", 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3603999999999997e-05 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:39.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363199+0000", 2024-09-14T06:59:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9394999999999997e-05 2024-09-14T06:59:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:39.703 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:39.703 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:39.705 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:59:39.705 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:39.703+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:39.805 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:39.805+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:39.849 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:39.849 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:39.849 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:39.849 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:39.858 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:59:39.906 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:39.906+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:40.000 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:40.000 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:40.000 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:40.000 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:40.000 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:40.006 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:40.006+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:40.008 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:59:40.108 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:40.107+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:40.112 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:40.112 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:40.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:40.112 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:40.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.112 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:53 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'35, mlcod=17'33)", 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.423281+0000", 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.687671883, 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00211137, 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423281+0000", 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423281+0000", 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423282+0000", 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0699999999999996e-07 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423345+0000", 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3468999999999998e-05 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423358+0000", 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2435999999999999e-05 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423363+0000", 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2850000000000002e-06 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423395+0000", 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1602e-05 2024-09-14T06:59:40.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423407+0000", 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2692e-05 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425359+0000", 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019521709999999999 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425392+0000", 2024-09-14T06:59:40.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3008e-05 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:54 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'36, mlcod=17'33)", 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.423853+0000", 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.687100211000001, 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0043481910000000004, 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423853+0000", 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423853+0000", 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423853+0000", 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8699999999999995e-07 2024-09-14T06:59:40.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423924+0000", 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1045999999999997e-05 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423933+0000", 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.49e-06 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423937+0000", 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8700000000000002e-06 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423978+0000", 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1625000000000002e-05 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.423993+0000", 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5104e-05 2024-09-14T06:59:40.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.428168+0000", 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041749129999999997 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.428201+0000", 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2555999999999998e-05 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.121 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:55 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'37, mlcod=17'33)", 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.424336+0000", 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.686616452999999, 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003882861, 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424336+0000", 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424336+0000", 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424337+0000", 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2099999999999997e-07 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424411+0000", 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3936999999999994e-05 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424421+0000", 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0328e-05 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424426+0000", 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2810000000000002e-06 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424461+0000", 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5244999999999999e-05 2024-09-14T06:59:40.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424475+0000", 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4164e-05 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.428204+0000", 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037293119999999998 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.428219+0000", 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5073e-05 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:56 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'38, mlcod=17'33)", 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.424920+0000", 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.686033092999999, 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061030700000000004, 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.126 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424920+0000", 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424920+0000", 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.424920+0000", 2024-09-14T06:59:40.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3799999999999997e-07 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425021+0000", 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000100388 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425029+0000", 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6219999999999998e-06 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425033+0000", 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0400000000000003e-06 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425082+0000", 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8511999999999999e-05 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.425099+0000", 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7354999999999999e-05 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.430989+0000", 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0058901739999999998 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.431023+0000", 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3441000000000001e-05 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:58 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'40, mlcod=17'38)", 2024-09-14T06:59:40.130 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:14.434359+0000", 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.676593764, 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002374223, 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434359+0000", 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434359+0000", 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434360+0000", 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6000000000000003e-07 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434365+0000", 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0509999999999996e-06 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434375+0000", 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0165e-05 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434381+0000", 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1380000000000004e-06 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434458+0000", 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6414000000000004e-05 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.434472+0000", 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4732999999999999e-05 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.436701+0000", 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022286739999999999 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:14.436733+0000", 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2388000000000002e-05 2024-09-14T06:59:40.134 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:92 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'66, mlcod=18'63)", 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.025527+0000", 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.085426097999999, 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.004154062, 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025527+0000", 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025527+0000", 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025527+0000", 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025531+0000", 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0899999999999998e-06 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025541+0000", 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7729999999999994e-06 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025547+0000", 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3569999999999997e-06 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025579+0000", 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.269e-05 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025592+0000", 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2799000000000001e-05 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.029639+0000", 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040465079999999999 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.029681+0000", 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2333000000000003e-05 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:93 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'67, mlcod=18'63)", 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.025869+0000", 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.085083879999999, 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0050062630000000004, 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.139 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025869+0000", 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025869+0000", 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025870+0000", 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1600000000000001e-07 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025932+0000", 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2438000000000001e-05 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025939+0000", 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1539999999999996e-06 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025944+0000", 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0150000000000003e-06 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025970+0000", 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5500999999999999e-05 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.025981+0000", 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0925e-05 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.030837+0000", 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048562249999999996 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.030875+0000", 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8488999999999997e-05 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.143 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:94 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'68, mlcod=18'64)", 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.026545+0000", 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.084408014000001, 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0073334020000000001, 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026545+0000", 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026545+0000", 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026545+0000", 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.01e-07 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026551+0000", 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2510000000000002e-06 2024-09-14T06:59:40.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026557+0000", 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0639999999999999e-06 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.026561+0000", 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3819999999999997e-06 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.029702+0000", 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031409540000000001 2024-09-14T06:59:40.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.029717+0000", 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5029e-05 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.033841+0000", 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041235330000000004 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.033878+0000", 2024-09-14T06:59:40.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7787999999999998e-05 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:96 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'70, mlcod=18'67)", 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.035205+0000", 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.075747543999999, 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0036047890000000002, 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035205+0000", 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035205+0000", 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035206+0000", 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6199999999999999e-07 2024-09-14T06:59:40.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035256+0000", 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9895999999999998e-05 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035263+0000", 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1069999999999999e-06 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035266+0000", 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.36e-06 2024-09-14T06:59:40.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035305+0000", 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9110999999999999e-05 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.035323+0000", 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8029e-05 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.038769+0000", 2024-09-14T06:59:40.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034452549999999999 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.038810+0000", 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1668999999999997e-05 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4204.0:98 1.0 e18/16 1:30306672:devicehealth::main.db.0000000000000000:head v 18'72, mlcod=18'70)", 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:15.042159+0000", 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.068794142000002, 2024-09-14T06:59:40.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002029229, 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042159+0000", 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042159+0000", 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042159+0000", 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8899999999999999e-07 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042163+0000", 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9859999999999996e-06 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042173+0000", 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.73e-06 2024-09-14T06:59:40.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042178+0000", 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.605e-06 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042219+0000", 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1415000000000002e-05 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.042239+0000", 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0072999999999999e-05 2024-09-14T06:59:40.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.044096+0000", 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018567989999999999 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:15.044188+0000", 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2032000000000002e-05 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2046 3.1f 3:f96ee6fa:::rbd_data.114fe955cd3f.0000000000000002:head [stat,truncate 1048576] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.352118+0000", 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75883451599999996, 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0026713129999999998, 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2046 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352118+0000", 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352118+0000", 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352120+0000", 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T06:59:40.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352123+0000", 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8550000000000004e-06 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352133+0000", 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3640000000000005e-06 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352138+0000", 2024-09-14T06:59:40.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.843e-06 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352180+0000", 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2716000000000002e-05 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352238+0000", 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7878999999999999e-05 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352354+0000", 2024-09-14T06:59:40.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000115727 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.353981+0000", 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0016274449999999999 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354722+0000", 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074045800000000004 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354759+0000", 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7404999999999998e-05 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354790+0000", 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0542e-05 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2039 3.10 e31/24 3:08f3caa9:::rbd_object_map.114fe955cd3f:head v 31'76, mlcod=31'73)", 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.352636+0000", 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75831728799999998, 2024-09-14T06:59:40.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048574830000000001, 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352636+0000", 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352636+0000", 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352636+0000", 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2100000000000002e-07 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352639+0000", 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8430000000000001e-06 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352645+0000", 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7359999999999998e-06 2024-09-14T06:59:40.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352648+0000", 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3000000000000002e-06 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352689+0000", 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1032999999999998e-05 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.352706+0000", 2024-09-14T06:59:40.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7450999999999999e-05 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357453+0000", 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0047468620000000001 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357493+0000", 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9836999999999998e-05 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2040 3.10 e31/24 3:08f3caa9:::rbd_object_map.114fe955cd3f:head v 31'77, mlcod=31'73)", 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.353628+0000", 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75732509100000001, 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0039000049999999998, 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.353628+0000", 2024-09-14T06:59:40.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.353628+0000", 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.353628+0000", 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4799999999999999e-07 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.353633+0000", 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1209999999999998e-06 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.353641+0000", 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1520000000000003e-06 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.353644+0000", 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.664e-06 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.353681+0000", 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6551000000000003e-05 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.353698+0000", 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7224e-05 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357496+0000", 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037974969999999999 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357528+0000", 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2348000000000003e-05 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2041 3.10 e31/24 3:08f3caa9:::rbd_object_map.114fe955cd3f:head v 31'78, mlcod=31'73)", 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.354247+0000", 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75670549099999995, 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033040130000000002, 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354247+0000", 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354247+0000", 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354248+0000", 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354251+0000", 2024-09-14T06:59:40.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1700000000000001e-06 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354257+0000", 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.879e-06 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354260+0000", 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9239999999999999e-06 2024-09-14T06:59:40.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354284+0000", 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4535000000000001e-05 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354296+0000", 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1779e-05 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357529+0000", 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032327419999999998 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.357551+0000", 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2608000000000001e-05 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2042 3.10 e31/24 3:08f3caa9:::rbd_object_map.114fe955cd3f:head v 31'79, mlcod=31'73)", 2024-09-14T06:59:40.176 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.354815+0000", 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75613817500000002, 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0058222589999999998, 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354815+0000", 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354815+0000", 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354815+0000", 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.37e-07 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354820+0000", 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3599999999999998e-06 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354828+0000", 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1729999999999995e-06 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354832+0000", 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5419999999999999e-06 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354868+0000", 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5781000000000002e-05 2024-09-14T06:59:40.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.354879+0000", 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1318000000000001e-05 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360612+0000", 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00573245 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360637+0000", 2024-09-14T06:59:40.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5097999999999999e-05 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2043 3.10 e31/24 3:08f3caa9:::rbd_object_map.114fe955cd3f:head v 31'80, mlcod=31'73)", 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.355331+0000", 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75562166500000005, 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053207039999999999, 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355331+0000", 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355331+0000", 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355332+0000", 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9799999999999999e-07 2024-09-14T06:59:40.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355335+0000", 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4199999999999999e-06 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355341+0000", 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.772e-06 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355345+0000", 2024-09-14T06:59:40.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.583e-06 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355376+0000", 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0794999999999999e-05 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.355387+0000", 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0655e-05 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360638+0000", 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0052513129999999996 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360652+0000", 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3767999999999999e-05 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2049 3.10 e31/24 3:08f3caa9:::rbd_object_map.114fe955cd3f:head v 31'81, mlcod=31'77)", 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358348+0000", 2024-09-14T06:59:40.185 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75260488199999998, 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048898129999999998, 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358348+0000", 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358348+0000", 2024-09-14T06:59:40.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358349+0000", 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9599999999999998e-07 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358355+0000", 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0240000000000003e-06 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358367+0000", 2024-09-14T06:59:40.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2135e-05 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358374+0000", 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8859999999999998e-06 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358428+0000", 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4418e-05 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358447+0000", 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.838e-05 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363199+0000", 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0047526560000000001 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363238+0000", 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8417999999999997e-05 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.189 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2053 3.10 e31/24 3:08f3caa9:::rbd_object_map.114fe955cd3f:head v 31'82, mlcod=31'77)", 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.360222+0000", 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.75073127399999995, 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003447636, 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360222+0000", 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360222+0000", 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360222+0000", 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7100000000000001e-07 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360228+0000", 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5960000000000003e-06 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360242+0000", 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4129e-05 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360249+0000", 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.883e-06 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360302+0000", 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2887999999999999e-05 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.360323+0000", 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1141e-05 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363647+0000", 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003323636 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363669+0000", 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2592e-05 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2057 3.10 e31/24 3:08f3caa9:::rbd_object_map.114fe955cd3f:head v 31'84, mlcod=31'79)", 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.363817+0000", 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.74713557399999997, 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029291410000000001, 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.194 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363817+0000", 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363817+0000", 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363818+0000", 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9499999999999998e-07 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363829+0000", 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0815000000000001e-05 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363836+0000", 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6629999999999993e-06 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363840+0000", 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9899999999999999e-06 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363877+0000", 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6408000000000001e-05 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.363888+0000", 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1802e-05 2024-09-14T06:59:40.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.366720+0000", 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028310750000000002 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.366747+0000", 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6993000000000001e-05 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2058 3.10 e31/24 3:08f3caa9:::rbd_object_map.114fe955cd3f:head v 31'85, mlcod=31'81)", 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.364610+0000", 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.74634281700000005, 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0045103950000000004, 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.364610+0000", 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.364610+0000", 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.364610+0000", 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7099999999999997e-07 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.364613+0000", 2024-09-14T06:59:40.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.289e-06 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.364619+0000", 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9100000000000002e-06 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.364622+0000", 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3790000000000002e-06 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.364655+0000", 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2978000000000002e-05 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.364667+0000", 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2133999999999999e-05 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.369099+0000", 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044318600000000001 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.369121+0000", 2024-09-14T06:59:40.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1474000000000001e-05 2024-09-14T06:59:40.203 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.203 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.203 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.203 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.203 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.203 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:40.204 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:59:40.209 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:40.208+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:40.310 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:40.309+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:40.348 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-14T06:59:40.349 INFO:teuthology.orchestra.run.smithi007.stdout:570 write 0xc152e2 thru 0xc1d7cd (0x84ec bytes) 2024-09-14T06:59:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:40.356 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:40.356 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:40.356 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:40.357 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:40.357 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:40.357 INFO:teuthology.orchestra.run.smithi007.stdout:574 write 0x9930c29 thru 0x993d565 (0xc93d bytes) 2024-09-14T06:59:40.361 INFO:teuthology.orchestra.run.smithi007.stdout:575 read 0x654d32c thru 0x655a672 (0xd347 bytes) 2024-09-14T06:59:40.362 INFO:teuthology.orchestra.run.smithi007.stdout:576 read 0x291463c thru 0x2920ff5 (0xc9ba bytes) 2024-09-14T06:59:40.362 INFO:teuthology.orchestra.run.smithi007.stdout:577 read 0x829d1da thru 0x82a3dd2 (0x6bf9 bytes) 2024-09-14T06:59:40.364 INFO:teuthology.orchestra.run.smithi007.stdout:579 trunc from 0xe09315e to 0xec3890c 2024-09-14T06:59:40.365 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:59:40.370 INFO:teuthology.orchestra.run.smithi007.stdout:580 write 0xcdcd3a8 thru 0xcdd1c17 (0x4870 bytes) 2024-09-14T06:59:40.375 INFO:teuthology.orchestra.run.smithi007.stdout:581 write 0x417ddb1 thru 0x418d086 (0xf2d6 bytes) 2024-09-14T06:59:40.380 INFO:teuthology.orchestra.run.smithi007.stdout:582 writesame 0x1a76012 thru 0x1a787f6 data_size 0x1(0x27e5 bytes) 2024-09-14T06:59:40.400 INFO:teuthology.orchestra.run.smithi007.stdout:583 trunc from 0xec3890c to 0x5dc9af1 2024-09-14T06:59:40.410 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:40.409+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:40.423 INFO:teuthology.orchestra.run.smithi007.stdout:585 write 0x3e3f87 thru 0x3f032d (0xc3a7 bytes) 2024-09-14T06:59:40.432 INFO:teuthology.orchestra.run.smithi007.stdout:587 trunc from 0x5dc9af1 to 0x35c2a08 2024-09-14T06:59:40.450 INFO:teuthology.orchestra.run.smithi007.stdout:589 flatten 2024-09-14T06:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:591 read 0x2dfb7f7 thru 0x2dfbe09 (0x613 bytes) 2024-09-14T06:59:40.511 INFO:teuthology.orchestra.run.smithi007.stdout:592 read 0x20d1eac thru 0x20db3da (0x952f bytes) 2024-09-14T06:59:40.511 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:40.510+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:40.512 INFO:teuthology.orchestra.run.smithi007.stdout:593 trunc from 0x35c2a08 to 0x4dba3e8 2024-09-14T06:59:40.517 INFO:teuthology.orchestra.run.smithi007.stdout:594 read 0xa79cbc thru 0xa82b38 (0x8e7d bytes) 2024-09-14T06:59:40.518 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:40.518 INFO:teuthology.orchestra.run.smithi007.stdout:596 read 0x441e844 thru 0x442c5ac (0xdd69 bytes) 2024-09-14T06:59:40.529 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:40.529 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:40.529 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:40.529 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:40.529 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:40.534 INFO:teuthology.orchestra.run.smithi007.stdout:597 write 0xeb26afd thru 0xeb2ad4e (0x4252 bytes) 2024-09-14T06:59:40.537 INFO:teuthology.orchestra.run.smithi007.stdout:598 write 0xca48739 thru 0xca49b70 (0x1438 bytes) 2024-09-14T06:59:40.538 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:59:40.540 INFO:teuthology.orchestra.run.smithi007.stdout:599 write 0x719dc0b thru 0x71a857f (0xa975 bytes) 2024-09-14T06:59:40.544 INFO:teuthology.orchestra.run.smithi007.stdout:600 writesame 0x395a494 thru 0x3968584 data_size 0x37(0xe0f1 bytes) 2024-09-14T06:59:40.547 INFO:teuthology.orchestra.run.smithi007.stdout:601 punch from 0x9c0d6d0 to 0x9c0e8b0, (0x11e0 bytes) 2024-09-14T06:59:40.548 INFO:teuthology.orchestra.run.smithi007.stdout:602 read 0x7c172e1 thru 0x7c183cf (0x10ef bytes) 2024-09-14T06:59:40.548 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:40.548 INFO:teuthology.orchestra.run.smithi007.stdout:605 write 0xc5adfc thru 0xc5b88b (0xa90 bytes) 2024-09-14T06:59:40.550 INFO:teuthology.orchestra.run.smithi007.stdout:606 read 0x9380efa thru 0x938d89d (0xc9a4 bytes) 2024-09-14T06:59:40.550 INFO:teuthology.orchestra.run.smithi007.stdout:607 write 0xdbedca8 thru 0xdbf4844 (0x6b9d bytes) 2024-09-14T06:59:40.554 INFO:teuthology.orchestra.run.smithi007.stdout:608 read 0x115aeaf thru 0x1162129 (0x727b bytes) 2024-09-14T06:59:40.554 INFO:teuthology.orchestra.run.smithi007.stdout:609 read 0x9d9f138 thru 0x9da6935 (0x77fe bytes) 2024-09-14T06:59:40.554 INFO:teuthology.orchestra.run.smithi007.stdout:612 writesame 0x2a13480 thru 0x2a1fdc9 data_size 0x5(0xc94a bytes) 2024-09-14T06:59:40.560 INFO:teuthology.orchestra.run.smithi007.stdout:613 write 0x3143c02 thru 0x314ae9e (0x729d bytes) 2024-09-14T06:59:40.563 INFO:teuthology.orchestra.run.smithi007.stdout:614 read 0xa04da1b thru 0xa05a052 (0xc638 bytes) 2024-09-14T06:59:40.563 INFO:teuthology.orchestra.run.smithi007.stdout:615 write 0x3d23a70 thru 0x3d3138b (0xd91c bytes) 2024-09-14T06:59:40.566 INFO:teuthology.orchestra.run.smithi007.stdout:617 write 0x9da2e65 thru 0x9dacecc (0xa068 bytes) 2024-09-14T06:59:40.570 INFO:teuthology.orchestra.run.smithi007.stdout:618 write 0xc30e08e thru 0xc31a38d (0xc300 bytes) 2024-09-14T06:59:40.582 INFO:teuthology.orchestra.run.smithi007.stdout:619 trunc from 0xeb2ad4f to 0x65d8227 2024-09-14T06:59:40.593 INFO:teuthology.orchestra.run.smithi007.stdout:620 writesame 0x36fffad thru 0x37017e9 data_size 0x11(0x183d bytes) 2024-09-14T06:59:40.600 INFO:teuthology.orchestra.run.smithi007.stdout:621 trunc from 0x65d8227 to 0x8f31171 2024-09-14T06:59:40.614 INFO:teuthology.orchestra.run.smithi007.stdout:622 write 0xbdfe11e thru 0xbe0dd73 (0xfc56 bytes) 2024-09-14T06:59:40.625 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:40.624+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:40.629 INFO:teuthology.orchestra.run.smithi007.stdout:623 trunc from 0xbe0dd74 to 0x149f80d 2024-09-14T06:59:40.650 INFO:teuthology.orchestra.run.smithi007.stdout:627 read 0xcd2047 thru 0xcda552 (0x850c bytes) 2024-09-14T06:59:40.651 INFO:teuthology.orchestra.run.smithi007.stdout:629 writesame 0x8a7344 thru 0x8b2194 data_size 0x7d(0xae51 bytes) 2024-09-14T06:59:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:630 writesame 0x83c8fdf thru 0x83d7300 data_size 0x6(0xe322 bytes) 2024-09-14T06:59:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:631 read 0x8059052 thru 0x805c5c5 (0x3574 bytes) 2024-09-14T06:59:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:632 punch from 0x49bcc97 to 0x49c9362, (0xc6cb bytes) 2024-09-14T06:59:40.665 INFO:teuthology.orchestra.run.smithi007.stdout:633 read 0x589aeb4 thru 0x58a339e (0x84eb bytes) 2024-09-14T06:59:40.678 INFO:teuthology.orchestra.run.smithi007.stdout:634 writesame 0xea9c7f9 thru 0xeaa2fd1 data_size 0xd(0x67d9 bytes) 2024-09-14T06:59:40.678 INFO:teuthology.orchestra.run.smithi007.stdout:636 writesame 0xb7c4475 thru 0xb7cebde data_size 0x9(0xa76a bytes) 2024-09-14T06:59:40.692 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:40.693 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:40.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:40.693 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:40.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.693 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T06:59:40.693 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:40.693 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.926784484999999, 2024-09-14T06:59:40.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T06:59:40.697 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.697 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T06:59:40.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:40.702 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.853665219, 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T06:59:40.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T06:59:40.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T06:59:40.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.8534085660000001, 2024-09-14T06:59:40.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T06:59:40.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T06:59:40.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.714 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:1845 3.5 e30/24 3:a1c743af:::rbd_data.114fe955cd3f.0000000000000005:head v 30'40, mlcod=30'38)", 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837741+0000", 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.853192339, 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0010615679999999999, 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837741+0000", 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837741+0000", 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837742+0000", 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5499999999999996e-07 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837851+0000", 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000109091 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837868+0000", 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7054e-05 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837875+0000", 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0709999999999997e-06 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837929+0000", 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4279e-05 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837952+0000", 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2504e-05 2024-09-14T06:59:40.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838766+0000", 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081464999999999999 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838802+0000", 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5964000000000003e-05 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.719 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.8271214919999998, 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:40.720 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T06:59:40.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T06:59:40.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.8269968780000001, 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.725 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:40.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:40.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:40.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:40.726 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T06:59:40.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.726 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.730 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "pg_update_log_missing(3.5 epoch 30/24 rep_tid 54 entries 30'41 (0'0) error 3:a1c743af:::rbd_data.114fe955cd3f.0000000000000005:head by client.4431.0:1877 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 30'39)", 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.864220+0000", 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.8267128800000001, 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081954700000000001, 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864220+0000", 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864220+0000", 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864221+0000", 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5600000000000005e-07 2024-09-14T06:59:40.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864228+0000", 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3069999999999997e-06 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864244+0000", 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6087999999999999e-05 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864258+0000", 2024-09-14T06:59:40.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4080000000000001e-05 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864459+0000", 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000201246 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.865040+0000", 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058016999999999997 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:1891 3.b e30/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 30'28, mlcod=30'26)", 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.886394+0000", 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.8045394560000001, 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00095844300000000001, 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.886394+0000", 2024-09-14T06:59:40.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.886394+0000", 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.886395+0000", 2024-09-14T06:59:40.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5349999999999999e-06 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.886425+0000", 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0318e-05 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.886446+0000", 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0826e-05 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.886474+0000", 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8132000000000001e-05 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.886549+0000", 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4821999999999996e-05 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.886567+0000", 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7655999999999998e-05 2024-09-14T06:59:40.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.887317+0000", 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000749813 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.887352+0000", 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5340999999999999e-05 2024-09-14T06:59:40.739 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1903 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,write 20253103~57310] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.915008+0000", 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7759247729999998, 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015901260000000001, 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:40.740 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1903 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915008+0000", 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915008+0000", 2024-09-14T06:59:40.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915010+0000", 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915035+0000", 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4938000000000001e-05 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915049+0000", 2024-09-14T06:59:40.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4208e-05 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915057+0000", 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3340000000000002e-06 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915110+0000", 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2451000000000003e-05 2024-09-14T06:59:40.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915171+0000", 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1116000000000002e-05 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915258+0000", 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7565999999999995e-05 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.915929+0000", 2024-09-14T06:59:40.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067060299999999998 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.916530+0000", 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060087300000000003 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.916563+0000", 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3500000000000001e-05 2024-09-14T06:59:40.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.916598+0000", 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5009999999999999e-05 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:1934 3.b e30/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 30'29, mlcod=30'27)", 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.943277+0000", 2024-09-14T06:59:40.746 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7476561589999999, 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00091252499999999997, 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.943277+0000", 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.943277+0000", 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.943278+0000", 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9999999999999996e-07 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.943297+0000", 2024-09-14T06:59:40.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9519000000000001e-05 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.943309+0000", 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1982e-05 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.943316+0000", 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8909999999999998e-06 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.943366+0000", 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9595999999999997e-05 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.943387+0000", 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1330000000000001e-05 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.944161+0000", 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00077435999999999998 2024-09-14T06:59:40.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.944189+0000", 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8047000000000001e-05 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2050 3.5 e31/24 3:a1c743af:::rbd_data.114fe955cd3f.0000000000000005:head v 31'45, mlcod=31'42)", 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358226+0000", 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.3327072119999999, 2024-09-14T06:59:40.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001276363, 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358226+0000", 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358226+0000", 2024-09-14T06:59:40.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358227+0000", 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358236+0000", 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7199999999999995e-06 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358250+0000", 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4426e-05 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358259+0000", 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7069999999999998e-06 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358324+0000", 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5671000000000002e-05 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358348+0000", 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3204e-05 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.359471+0000", 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001123799 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.359502+0000", 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0806000000000003e-05 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.755 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.3323777159999999, 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T06:59:40.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T06:59:40.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T06:59:40.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T06:59:40.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T06:59:40.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2047 3.12 e32/31 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head v 32'67, mlcod=0'0)", 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.285952+0000", 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40498130100000002, 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001098471, 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.761 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.762 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.285952+0000", 2024-09-14T06:59:40.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.285952+0000", 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.285952+0000", 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.37e-07 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.285959+0000", 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4629999999999998e-06 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.285968+0000", 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9139999999999997e-06 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.285977+0000", 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0059999999999998e-06 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.286021+0000", 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3763000000000001e-05 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.286037+0000", 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6731e-05 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287024+0000", 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00098652099999999997 2024-09-14T06:59:40.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287050+0000", 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6336e-05 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2051 3.12 e32/31 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head v 32'69, mlcod=0'0)", 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.287468+0000", 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40346531200000002, 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001003599, 2024-09-14T06:59:40.766 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287468+0000", 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287468+0000", 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287468+0000", 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3300000000000002e-07 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287480+0000", 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1671000000000001e-05 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287487+0000", 2024-09-14T06:59:40.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1600000000000001e-06 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287492+0000", 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3789999999999997e-06 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287527+0000", 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4295000000000003e-05 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.287539+0000", 2024-09-14T06:59:40.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2194999999999999e-05 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.288435+0000", 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00089607800000000002 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.288471+0000", 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6288000000000002e-05 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2355 3.b e32/24 3:d41a69af:::rbd_data.114fe2819363.0000000000000008:head v 32'35, mlcod=32'33)", 2024-09-14T06:59:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.427787+0000", 2024-09-14T06:59:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.26314610799999999, 2024-09-14T06:59:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001101877, 2024-09-14T06:59:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.771 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.427787+0000", 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.427787+0000", 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.427788+0000", 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2550000000000001e-06 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.427829+0000", 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0633000000000002e-05 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.427845+0000", 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6507e-05 2024-09-14T06:59:40.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.427854+0000", 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5900000000000008e-06 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.427915+0000", 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1481000000000003e-05 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.427931+0000", 2024-09-14T06:59:40.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6158999999999999e-05 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.428846+0000", 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0009144 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.428889+0000", 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2852000000000002e-05 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.776 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2364 3.7 e32/24 3:e6755b94:::rbd_data.114fe2819363.00000000000000b9:head v 32'39, mlcod=32'37)", 2024-09-14T06:59:40.776 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.434667+0000", 2024-09-14T06:59:40.776 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.25626558799999999, 2024-09-14T06:59:40.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001473147, 2024-09-14T06:59:40.776 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.776 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.776 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.776 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.434667+0000", 2024-09-14T06:59:40.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.434667+0000", 2024-09-14T06:59:40.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.434668+0000", 2024-09-14T06:59:40.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4e-07 2024-09-14T06:59:40.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.434685+0000", 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6600999999999999e-05 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.434696+0000", 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1206000000000001e-05 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.434704+0000", 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7950000000000008e-06 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.434760+0000", 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5849e-05 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.434777+0000", 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7778000000000002e-05 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.436090+0000", 2024-09-14T06:59:40.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001312368 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.436141+0000", 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0710000000000001e-05 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3038 3.b e32/24 3:d5822750:::rbd_data.114fe2819363.000000000000003d:head v 32'36, mlcod=32'34)", 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.488972+0000", 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.201961007, 2024-09-14T06:59:40.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0010341149999999999, 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.488972+0000", 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.488972+0000", 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.488973+0000", 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1399999999999996e-07 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.488998+0000", 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4831999999999999e-05 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.489010+0000", 2024-09-14T06:59:40.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2532e-05 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.489017+0000", 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2609999999999996e-06 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.489083+0000", 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6067000000000003e-05 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.489103+0000", 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9735e-05 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.489967+0000", 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00086406600000000003 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.490006+0000", 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8908000000000001e-05 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3175 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'37, mlcod=32'35)", 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.495904+0000", 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.195028862, 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00090167700000000003, 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495904+0000", 2024-09-14T06:59:40.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495904+0000", 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495905+0000", 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9699999999999994e-07 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495923+0000", 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7453999999999999e-05 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495937+0000", 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4754e-05 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495945+0000", 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0760000000000003e-06 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.496014+0000", 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8722999999999999e-05 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.496035+0000", 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1056e-05 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.496750+0000", 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00071473399999999996 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.496806+0000", 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5883e-05 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3320 3.15 e32/24 3:aa29f3eb:::rbd_data.114fe2819363.00000000000001b8:head v 32'1, mlcod=0'0)", 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.553310+0000", 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.13762321199999999, 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081435500000000001, 2024-09-14T06:59:40.790 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.553310+0000", 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.553310+0000", 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.553311+0000", 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.22e-06 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.553335+0000", 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3771999999999999e-05 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.553350+0000", 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5407e-05 2024-09-14T06:59:40.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.553358+0000", 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4460000000000004e-06 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.553406+0000", 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8665000000000002e-05 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.553427+0000", 2024-09-14T06:59:40.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0526999999999998e-05 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.554081+0000", 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000654533 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.554124+0000", 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2784999999999998e-05 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3343 3.15 e32/24 3:aa29f3eb:::rbd_data.114fe2819363.00000000000001b8:head v 32'2, mlcod=0'0)", 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.585438+0000", 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.105495227, 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001057555, 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.585438+0000", 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.585438+0000", 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.585439+0000", 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.192e-06 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.585446+0000", 2024-09-14T06:59:40.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0450000000000003e-06 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.585461+0000", 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4778e-05 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.585469+0000", 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7299999999999994e-06 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:40.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.585543+0000", 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3987999999999997e-05 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.585565+0000", 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1087e-05 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.586458+0000", 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00089314400000000001 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:40.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.586495+0000", 2024-09-14T06:59:40.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7591e-05 2024-09-14T06:59:40.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.799 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:40.799 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:40.799 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:40.801 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:59:40.802 INFO:teuthology.orchestra.run.smithi007.stdout:637 trunc from 0xeaa2fd2 to 0x593e11d 2024-09-14T06:59:40.802 INFO:teuthology.orchestra.run.smithi007.stdout:639 read 0x30e66bd thru 0x30ee54f (0x7e93 bytes) 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:640 writesame 0x79d84f1 thru 0x79e61cc data_size 0x6e(0xdcdc bytes) 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:641 trunc from 0x79e61cd to 0x72e53b6 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:644 punch from 0x200168 to 0x206a5b, (0x68f3 bytes) 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:645 trunc from 0x72e53b6 to 0x4c841b7 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:646 write 0xdba66b9 thru 0xdbaddcd (0x7715 bytes) 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:647 read 0x82bd4ca thru 0x82bfefb (0x2a32 bytes) 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:648 writesame 0xa9943ef thru 0xa99b6c4 data_size 0x1(0x72d6 bytes) 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:649 trunc from 0xdbaddce to 0x8643ca8 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:650 read 0x21cfede thru 0x21d76b7 (0x77da bytes) 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:651 write 0x1a62080 thru 0x1a719d4 (0xf955 bytes) 2024-09-14T06:59:40.803 INFO:teuthology.orchestra.run.smithi007.stdout:652 writesame 0x603b038 thru 0x6049628 data_size 0x3d(0xe5f1 bytes) 2024-09-14T06:59:40.804 INFO:teuthology.orchestra.run.smithi007.stdout:654 write 0x6c34fb9 thru 0x6c42b02 (0xdb4a bytes) 2024-09-14T06:59:40.804 INFO:teuthology.orchestra.run.smithi007.stdout:655 trunc from 0x8643ca8 to 0xb16a450 2024-09-14T06:59:40.804 INFO:teuthology.orchestra.run.smithi007.stdout:656 punch from 0x9968d87 to 0x996a9b2, (0x1c2b bytes) 2024-09-14T06:59:40.804 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:40.800+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:40.804 INFO:teuthology.orchestra.run.smithi007.stdout:657 trunc from 0xb16a450 to 0x8200168 2024-09-14T06:59:40.807 INFO:teuthology.orchestra.run.smithi007.stdout:661 read 0x17fd380 thru 0x180be40 (0xeac1 bytes) 2024-09-14T06:59:40.818 INFO:teuthology.orchestra.run.smithi007.stdout:662 writesame 0xcd4389b thru 0xcd47c41 data_size 0x3(0x43a7 bytes) 2024-09-14T06:59:40.818 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:40.818 INFO:teuthology.orchestra.run.smithi007.stdout:665 writesame 0x51c112a thru 0x51cb942 data_size 0x1(0xa819 bytes) 2024-09-14T06:59:40.838 INFO:teuthology.orchestra.run.smithi007.stdout:666 read 0x78a0a85 thru 0x78a2370 (0x18ec bytes) 2024-09-14T06:59:40.838 INFO:teuthology.orchestra.run.smithi007.stdout:668 read 0xa04763d thru 0xa047c89 (0x64d bytes) 2024-09-14T06:59:40.840 INFO:teuthology.orchestra.run.smithi007.stdout:669 trunc from 0xcd47c42 to 0xe3f8d1c 2024-09-14T06:59:40.848 INFO:teuthology.orchestra.run.smithi007.stdout:670 write 0x728858 thru 0x72e4f0 (0x5c99 bytes) 2024-09-14T06:59:40.851 INFO:teuthology.orchestra.run.smithi007.stdout:671 read 0x9543d43 thru 0x9549c2a (0x5ee8 bytes) 2024-09-14T06:59:40.851 INFO:teuthology.orchestra.run.smithi007.stdout:672 write 0x500c555 thru 0x501190f (0x53bb bytes) 2024-09-14T06:59:40.854 INFO:teuthology.orchestra.run.smithi007.stdout:675 read 0xcb8d4d6 thru 0xcb91d7b (0x48a6 bytes) 2024-09-14T06:59:40.854 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:40.854 INFO:teuthology.orchestra.run.smithi007.stdout:677 punch from 0xc888ef0 to 0xc88c9ef, (0x3aff bytes) 2024-09-14T06:59:40.860 INFO:teuthology.orchestra.run.smithi007.stdout:678 write 0xee2aefb thru 0xee357d9 (0xa8df bytes) 2024-09-14T06:59:40.864 INFO:teuthology.orchestra.run.smithi007.stdout:679 writesame 0xb4de62f thru 0xb4e1819 data_size 0x1(0x31eb bytes) 2024-09-14T06:59:40.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:59:40.869 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout:681 punch from 0x3c08407 to 0x3c0c4ad, (0x40a6 bytes) 2024-09-14T06:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout:682 read 0x6da8f3e thru 0x6daf443 (0x6506 bytes) 2024-09-14T06:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout:683 punch from 0x48b976d to 0x48bfcc8, (0x655b bytes) 2024-09-14T06:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout:684 read 0x5b0dc35 thru 0x5b0f044 (0x1410 bytes) 2024-09-14T06:59:40.870 INFO:teuthology.orchestra.run.smithi007.stdout:685 punch from 0x6d6783d to 0x6d72c91, (0xb454 bytes) 2024-09-14T06:59:40.871 INFO:teuthology.orchestra.run.smithi007.stdout:686 write 0x15630d5 thru 0x1569775 (0x66a1 bytes) 2024-09-14T06:59:40.872 INFO:teuthology.orchestra.run.smithi007.stdout:687 read 0x9bc9b16 thru 0x9bcf447 (0x5932 bytes) 2024-09-14T06:59:40.873 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:40.873 INFO:teuthology.orchestra.run.smithi007.stdout:690 clone 4 order 20 su 524288 sc 13 2024-09-14T06:59:40.901 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:40.901+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:40.946 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:40.946 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:40.946 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:40.946 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:40.955 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:59:40.991 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:40.992 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:41.002 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:41.001+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:41.009 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:41.025 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:41.041 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:41.042 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:41.058 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:41.076 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:41.076 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:41.093 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:41.093 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:41.100 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:41.100 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:41.100 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:41.101 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:41.101 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:41.103 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:41.102+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:41.109 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:59:41.110 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:41.110 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:41.203 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:41.203+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:41.252 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:41.253 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:41.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:59:41.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:59:41.305 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:41.304+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:41.379 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:41.379 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:41.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:59:41.406 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:41.405+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:41.507 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:41.506+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:41.508 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-14T06:59:41.512 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:41.512 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:41.516 INFO:teuthology.orchestra.run.smithi007.stdout:691 write 0xdb16d7a thru 0xdb1e30e (0x7595 bytes) 2024-09-14T06:59:41.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:59:41.523 INFO:teuthology.orchestra.run.smithi007.stdout:693 read 0x1f80337 thru 0x1f86460 (0x612a bytes) 2024-09-14T06:59:41.524 INFO:teuthology.orchestra.run.smithi007.stdout:694 writesame 0x110d0e0 thru 0x1113592 data_size 0xd(0x64b3 bytes) 2024-09-14T06:59:41.527 INFO:teuthology.orchestra.run.smithi007.stdout:695 writesame 0x2dac12f thru 0x2db54b8 data_size 0x5(0x938a bytes) 2024-09-14T06:59:41.533 INFO:teuthology.orchestra.run.smithi007.stdout:698 punch from 0xcc588ab to 0xcc6581b, (0xcf70 bytes) 2024-09-14T06:59:41.542 INFO:teuthology.orchestra.run.smithi007.stdout:699 trunc from 0xee357da to 0x73f9e96 2024-09-14T06:59:41.558 INFO:teuthology.orchestra.run.smithi007.stdout:700 punch from 0x9397ea to 0x939d42, (0x558 bytes) 2024-09-14T06:59:41.573 INFO:teuthology.orchestra.run.smithi007.stdout:702 writesame 0xe9caa2f thru 0xe9d7c58 data_size 0x62(0xd22a bytes) 2024-09-14T06:59:41.573 INFO:teuthology.orchestra.run.smithi007.stdout:703 writesame 0xa27f478 thru 0xa282a1d data_size 0x7(0x35a6 bytes) 2024-09-14T06:59:41.577 INFO:teuthology.orchestra.run.smithi007.stdout:704 write 0x45121c thru 0x45b39c (0xa181 bytes) 2024-09-14T06:59:41.581 INFO:teuthology.orchestra.run.smithi007.stdout:706 write 0x60ae0f8 thru 0x60b8563 (0xa46c bytes) 2024-09-14T06:59:41.597 INFO:teuthology.orchestra.run.smithi007.stdout:707 trunc from 0xe9d7c59 to 0x3902c0f 2024-09-14T06:59:41.608 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:41.607+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:41.622 INFO:teuthology.orchestra.run.smithi007.stdout:708 read 0x435e84 thru 0x444391 (0xe50e bytes) 2024-09-14T06:59:41.635 INFO:teuthology.orchestra.run.smithi007.stdout:710 write 0x9d42b88 thru 0x9d4ae96 (0x830f bytes) 2024-09-14T06:59:41.638 INFO:teuthology.orchestra.run.smithi007.stdout:712 write 0x2427dca thru 0x2432583 (0xa7ba bytes) 2024-09-14T06:59:41.642 INFO:teuthology.orchestra.run.smithi007.stdout:713 read 0x5932ad2 thru 0x593a094 (0x75c3 bytes) 2024-09-14T06:59:41.642 INFO:teuthology.orchestra.run.smithi007.stdout:714 punch from 0x3343ed0 to 0x334ca69, (0x8b99 bytes) 2024-09-14T06:59:41.642 INFO:teuthology.orchestra.run.smithi007.stdout:715 punch from 0x1c6e2d to 0x1cb54b, (0x471e bytes) 2024-09-14T06:59:41.642 INFO:teuthology.orchestra.run.smithi007.stdout:716 write 0x294ea2c thru 0x2959937 (0xaf0c bytes) 2024-09-14T06:59:41.645 INFO:teuthology.orchestra.run.smithi007.stdout:717 writesame 0x8a56a1 thru 0x8af293 data_size 0x25(0x9bf3 bytes) 2024-09-14T06:59:41.649 INFO:teuthology.orchestra.run.smithi007.stdout:718 write 0x13c8a52 thru 0x13d59fa (0xcfa9 bytes) 2024-09-14T06:59:41.660 INFO:teuthology.orchestra.run.smithi007.stdout:720 write 0xa510e7e thru 0xa512f3a (0x20bd bytes) 2024-09-14T06:59:41.664 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:41.664 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:41.673 INFO:teuthology.orchestra.run.smithi007.stdout:721 trunc from 0xa512f3b to 0x379a1a4 2024-09-14T06:59:41.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:59:41.689 INFO:teuthology.orchestra.run.smithi007.stdout:722 punch from 0x2d5dd05 to 0x2d6b2cb, (0xd5c6 bytes) 2024-09-14T06:59:41.689 INFO:teuthology.orchestra.run.smithi007.stdout:725 write 0x1838731 thru 0x1839665 (0xf35 bytes) 2024-09-14T06:59:41.700 INFO:teuthology.orchestra.run.smithi007.stdout:726 write 0x5d42aa1 thru 0x5d4c67c (0x9bdc bytes) 2024-09-14T06:59:41.703 INFO:teuthology.orchestra.run.smithi007.stdout:727 write 0x3fc085a thru 0x3fc8f96 (0x873d bytes) 2024-09-14T06:59:41.709 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:41.708+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:41.714 INFO:teuthology.orchestra.run.smithi007.stdout:729 writesame 0x83f1eec thru 0x83f2161 data_size 0x12(0x276 bytes) 2024-09-14T06:59:41.727 INFO:teuthology.orchestra.run.smithi007.stdout:730 write 0xe02f343 thru 0xe032d08 (0x39c6 bytes) 2024-09-14T06:59:41.731 INFO:teuthology.orchestra.run.smithi007.stdout:732 write 0x2e9734b thru 0x2e9913e (0x1df4 bytes) 2024-09-14T06:59:41.733 INFO:teuthology.orchestra.run.smithi007.stdout:733 write 0x7d9c19b thru 0x7dab621 (0xf487 bytes) 2024-09-14T06:59:41.736 INFO:teuthology.orchestra.run.smithi007.stdout:734 writesame 0x827842 thru 0x82cb7b data_size 0x6(0x533a bytes) 2024-09-14T06:59:41.741 INFO:teuthology.orchestra.run.smithi007.stdout:735 write 0x7377b59 thru 0x73782c0 (0x768 bytes) 2024-09-14T06:59:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:736 punch from 0xb7b9499 to 0xb7bad91, (0x18f8 bytes) 2024-09-14T06:59:41.744 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.748 INFO:teuthology.orchestra.run.smithi007.stdout:738 trunc from 0xe032d09 to 0xa80a0e1 2024-09-14T06:59:41.758 INFO:teuthology.orchestra.run.smithi007.stdout:739 write 0x38cf21c thru 0x38d71c0 (0x7fa5 bytes) 2024-09-14T06:59:41.762 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.762 INFO:teuthology.orchestra.run.smithi007.stdout:745 punch from 0x3329667 to 0x332d200, (0x3b99 bytes) 2024-09-14T06:59:41.762 INFO:teuthology.orchestra.run.smithi007.stdout:746 read 0x30aedbc thru 0x30b868d (0x98d2 bytes) 2024-09-14T06:59:41.763 INFO:teuthology.orchestra.run.smithi007.stdout:747 read 0x61f4967 thru 0x6204000 (0xf69a bytes) 2024-09-14T06:59:41.763 INFO:teuthology.orchestra.run.smithi007.stdout:749 read 0x364bdb6 thru 0x364da42 (0x1c8d bytes) 2024-09-14T06:59:41.763 INFO:teuthology.orchestra.run.smithi007.stdout:751 punch from 0x4f80aae to 0x4f88d36, (0x8288 bytes) 2024-09-14T06:59:41.767 INFO:teuthology.orchestra.run.smithi007.stdout:752 trunc from 0xa80a0e1 to 0x645fd22 2024-09-14T06:59:41.777 INFO:teuthology.orchestra.run.smithi007.stdout:753 writesame 0x14c041e thru 0x14c243b data_size 0x2(0x201e bytes) 2024-09-14T06:59:41.782 INFO:teuthology.orchestra.run.smithi007.stdout:754 read 0x52d4368 thru 0x52d96f5 (0x538e bytes) 2024-09-14T06:59:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:755 read 0xbfdc9d thru 0xc09c04 (0xbf68 bytes) 2024-09-14T06:59:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:756 read 0x3bf069 thru 0x3c29fb (0x3993 bytes) 2024-09-14T06:59:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:757 punch from 0x566c746 to 0x5671629, (0x4ee3 bytes) 2024-09-14T06:59:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:760 read 0x1e6ebad thru 0x1e7d56c (0xe9c0 bytes) 2024-09-14T06:59:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:764 read 0x3facc3d thru 0x3fb6ea0 (0xa264 bytes) 2024-09-14T06:59:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:765 punch from 0x39b32b3 to 0x39b948b, (0x61d8 bytes) 2024-09-14T06:59:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:767 punch from 0x40e5020 to 0x40f2da1, (0xdd81 bytes) 2024-09-14T06:59:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:768 writesame 0x6273211 thru 0x6279f01 data_size 0x1(0x6cf1 bytes) 2024-09-14T06:59:41.794 INFO:teuthology.orchestra.run.smithi007.stdout:769 read 0x56ef702 thru 0x56f82e3 (0x8be2 bytes) 2024-09-14T06:59:41.794 INFO:teuthology.orchestra.run.smithi007.stdout:770 punch from 0x2fea945 to 0x2fec0e2, (0x179d bytes) 2024-09-14T06:59:41.800 INFO:teuthology.orchestra.run.smithi007.stdout:771 write 0x6f50839 thru 0x6f5ce45 (0xc60d bytes) 2024-09-14T06:59:41.809 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:41.809+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:41.814 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:41.814 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:41.816 INFO:teuthology.orchestra.run.smithi007.stdout:772 write 0xe46218a thru 0xe46c8ea (0xa761 bytes) 2024-09-14T06:59:41.820 INFO:teuthology.orchestra.run.smithi007.stdout:773 read 0x5a5c775 thru 0x5a63386 (0x6c12 bytes) 2024-09-14T06:59:41.820 INFO:teuthology.orchestra.run.smithi007.stdout:775 write 0xdab4b38 thru 0xdabbfa2 (0x746b bytes) 2024-09-14T06:59:41.823 INFO:teuthology.orchestra.run.smithi007.stdout:776 read 0x1c82d20 thru 0x1c90020 (0xd301 bytes) 2024-09-14T06:59:41.823 INFO:teuthology.orchestra.run.smithi007.stdout:777 write 0x9b6616d thru 0x9b66642 (0x4d6 bytes) 2024-09-14T06:59:41.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:59:41.826 INFO:teuthology.orchestra.run.smithi007.stdout:778 punch from 0xca93bcc to 0xca99c56, (0x608a bytes) 2024-09-14T06:59:41.826 INFO:teuthology.orchestra.run.smithi007.stdout:779 read 0x68a7206 thru 0x68afac2 (0x88bd bytes) 2024-09-14T06:59:41.827 INFO:teuthology.orchestra.run.smithi007.stdout:780 writesame 0xc1d84c7 thru 0xc1db3ee data_size 0xc(0x2f28 bytes) 2024-09-14T06:59:41.829 INFO:teuthology.orchestra.run.smithi007.stdout:781 read 0x3552ebe thru 0x355b14c (0x828f bytes) 2024-09-14T06:59:41.830 INFO:teuthology.orchestra.run.smithi007.stdout:782 read 0x3fec33f thru 0x3ff9773 (0xd435 bytes) 2024-09-14T06:59:41.831 INFO:teuthology.orchestra.run.smithi007.stdout:783 writesame 0x1fa35e3 thru 0x1fa5b57 data_size 0x1(0x2575 bytes) 2024-09-14T06:59:41.837 INFO:teuthology.orchestra.run.smithi007.stdout:784 write 0x5f6b8ec thru 0x5f774b1 (0xbbc6 bytes) 2024-09-14T06:59:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:785 punch from 0x11bb50f to 0x11c68fa, (0xb3eb bytes) 2024-09-14T06:59:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:788 read 0x5379e54 thru 0x5384511 (0xa6be bytes) 2024-09-14T06:59:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:789 read 0xaa454f6 thru 0xaa494b5 (0x3fc0 bytes) 2024-09-14T06:59:41.841 INFO:teuthology.orchestra.run.smithi007.stdout:790 read 0x7cf50bc thru 0x7cf8072 (0x2fb7 bytes) 2024-09-14T06:59:41.842 INFO:teuthology.orchestra.run.smithi007.stdout:791 write 0x417de3f thru 0x418c887 (0xea49 bytes) 2024-09-14T06:59:41.852 INFO:teuthology.orchestra.run.smithi007.stdout:793 trunc from 0xe46c8eb to 0x8f99b12 2024-09-14T06:59:41.876 INFO:teuthology.orchestra.run.smithi007.stdout:796 write 0xdce2501 thru 0xdce56e7 (0x31e7 bytes) 2024-09-14T06:59:41.883 INFO:teuthology.orchestra.run.smithi007.stdout:797 trunc from 0xdce56e8 to 0xad60b21 2024-09-14T06:59:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:798 trunc from 0xad60b21 to 0xbf45dae 2024-09-14T06:59:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:799 read 0x4b0744d thru 0x4b0ad4a (0x38fe bytes) 2024-09-14T06:59:41.899 INFO:teuthology.orchestra.run.smithi007.stdout:800 writesame 0x86f9299 thru 0x870465c data_size 0xb1(0xb3c4 bytes) 2024-09-14T06:59:41.903 INFO:teuthology.orchestra.run.smithi007.stdout:801 write 0xb4b26c3 thru 0xb4b418c (0x1aca bytes) 2024-09-14T06:59:41.906 INFO:teuthology.orchestra.run.smithi007.stdout:804 write 0x67390f8 thru 0x67475ed (0xe4f6 bytes) 2024-09-14T06:59:41.910 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:41.909+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:41.910 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.910 INFO:teuthology.orchestra.run.smithi007.stdout:806 read 0x5ac6544 thru 0x5ad0957 (0xa414 bytes) 2024-09-14T06:59:41.910 INFO:teuthology.orchestra.run.smithi007.stdout:808 writesame 0xba09626 thru 0xba178e7 data_size 0x4b(0xe2c2 bytes) 2024-09-14T06:59:41.913 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.913 INFO:teuthology.orchestra.run.smithi007.stdout:811 write 0x2f21ded thru 0x2f26615 (0x4829 bytes) 2024-09-14T06:59:41.916 INFO:teuthology.orchestra.run.smithi007.stdout:812 writesame 0x60abac5 thru 0x60b127a data_size 0x67(0x57b6 bytes) 2024-09-14T06:59:41.917 INFO:teuthology.orchestra.run.smithi007.stdout:813 writesame 0x3fa6a1e thru 0x3fb2cd7 data_size 0x19(0xc2ba bytes) 2024-09-14T06:59:41.922 INFO:teuthology.orchestra.run.smithi007.stdout:814 read 0xac09f32 thru 0xac117c9 (0x7898 bytes) 2024-09-14T06:59:41.922 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.922 INFO:teuthology.orchestra.run.smithi007.stdout:816 read 0x63f7bf1 thru 0x6402f6a (0xb37a bytes) 2024-09-14T06:59:41.922 INFO:teuthology.orchestra.run.smithi007.stdout:817 read 0xbcb9b1c thru 0xbcbc728 (0x2c0d bytes) 2024-09-14T06:59:41.922 INFO:teuthology.orchestra.run.smithi007.stdout:818 writesame 0x85e27fb thru 0x85ee927 data_size 0x1(0xc12d bytes) 2024-09-14T06:59:41.940 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.940 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.942 INFO:teuthology.orchestra.run.smithi007.stdout:821 trunc from 0xbf45dae to 0xd1afe20 2024-09-14T06:59:41.946 INFO:teuthology.orchestra.run.smithi007.stdout:822 read 0x799fa86 thru 0x79ace39 (0xd3b4 bytes) 2024-09-14T06:59:41.946 INFO:teuthology.orchestra.run.smithi007.stdout:823 write 0xb6e59ba thru 0xb6ec6ce (0x6d15 bytes) 2024-09-14T06:59:41.950 INFO:teuthology.orchestra.run.smithi007.stdout:824 read 0x9f34dff thru 0x9f3da1d (0x8c1f bytes) 2024-09-14T06:59:41.950 INFO:teuthology.orchestra.run.smithi007.stdout:826 read 0x64ab9b7 thru 0x64b19a1 (0x5feb bytes) 2024-09-14T06:59:41.950 INFO:teuthology.orchestra.run.smithi007.stdout:827 writesame 0x87ada68 thru 0x87ba0c5 data_size 0x1(0xc65e bytes) 2024-09-14T06:59:41.953 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:41.953 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:41.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:59:41.972 INFO:teuthology.orchestra.run.smithi007.stdout:828 write 0xcd2119b thru 0xcd27f4c (0x6db2 bytes) 2024-09-14T06:59:41.975 INFO:teuthology.orchestra.run.smithi007.stdout:829 punch from 0x9f0049a to 0x9f08b0f, (0x8675 bytes) 2024-09-14T06:59:41.977 INFO:teuthology.orchestra.run.smithi007.stdout:830 trunc from 0xd1afe20 to 0xbc67b5c 2024-09-14T06:59:41.995 INFO:teuthology.orchestra.run.smithi007.stdout:832 writesame 0xcafc2f5 thru 0xcb09d60 data_size 0xe(0xda6c bytes) 2024-09-14T06:59:41.996 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.996 INFO:teuthology.orchestra.run.smithi007.stdout:834 read 0x9289dbb thru 0x928ed2e (0x4f74 bytes) 2024-09-14T06:59:41.996 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.996 INFO:teuthology.orchestra.run.smithi007.stdout:839 write 0x3fdda21 thru 0x3fed366 (0xf946 bytes) 2024-09-14T06:59:41.997 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.997 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:41.997 INFO:teuthology.orchestra.run.smithi007.stdout:843 read 0x7a1e942 thru 0x7a26bc8 (0x8287 bytes) 2024-09-14T06:59:41.997 INFO:teuthology.orchestra.run.smithi007.stdout:846 read 0x482e7ec thru 0x483da1b (0xf230 bytes) 2024-09-14T06:59:41.998 INFO:teuthology.orchestra.run.smithi007.stdout:847 read 0x430d300 thru 0x431119f (0x3ea0 bytes) 2024-09-14T06:59:41.998 INFO:teuthology.orchestra.run.smithi007.stdout:848 writesame 0x6d07cfd thru 0x6d0f02e data_size 0xf(0x7332 bytes) 2024-09-14T06:59:42.001 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:42.001 INFO:teuthology.orchestra.run.smithi007.stdout:850 read 0x645a3fc thru 0x645dc17 (0x381c bytes) 2024-09-14T06:59:42.001 INFO:teuthology.orchestra.run.smithi007.stdout:851 read 0x1a80a1 thru 0x1a919c (0x10fc bytes) 2024-09-14T06:59:42.002 INFO:teuthology.orchestra.run.smithi007.stdout:852 punch from 0x3201ed0 to 0x320bfdf, (0xa10f bytes) 2024-09-14T06:59:42.002 INFO:teuthology.orchestra.run.smithi007.stdout:853 read 0x85970bd thru 0x859e69d (0x75e1 bytes) 2024-09-14T06:59:42.002 INFO:teuthology.orchestra.run.smithi007.stdout:854 write 0x2f28bac thru 0x2f2cff1 (0x4446 bytes) 2024-09-14T06:59:42.004 INFO:teuthology.orchestra.run.smithi007.stdout:855 read 0x4304a60 thru 0x43109d7 (0xbf78 bytes) 2024-09-14T06:59:42.004 INFO:teuthology.orchestra.run.smithi007.stdout:856 punch from 0xae11ed5 to 0xae1bf75, (0xa0a0 bytes) 2024-09-14T06:59:42.004 INFO:teuthology.orchestra.run.smithi007.stdout:860 read 0x14edd05 thru 0x14f42f9 (0x65f5 bytes) 2024-09-14T06:59:42.004 INFO:teuthology.orchestra.run.smithi007.stdout:861 read 0x151db33 thru 0x151e765 (0xc33 bytes) 2024-09-14T06:59:42.005 INFO:teuthology.orchestra.run.smithi007.stdout:862 write 0x826ed79 thru 0x827c5ef (0xd877 bytes) 2024-09-14T06:59:42.008 INFO:teuthology.orchestra.run.smithi007.stdout:863 writesame 0x487cd92 thru 0x488be13 data_size 0x83(0xf082 bytes) 2024-09-14T06:59:42.011 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:42.010+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:42.012 INFO:teuthology.orchestra.run.smithi007.stdout:864 punch from 0x8c8fdce to 0x8c9921e, (0x9450 bytes) 2024-09-14T06:59:42.012 INFO:teuthology.orchestra.run.smithi007.stdout:866 punch from 0x1cc1da8 to 0x1cc4714, (0x296c bytes) 2024-09-14T06:59:42.012 INFO:teuthology.orchestra.run.smithi007.stdout:867 write 0x78c208c thru 0x78c36b0 (0x1625 bytes) 2024-09-14T06:59:42.026 INFO:teuthology.orchestra.run.smithi007.stdout:868 trunc from 0xcb09d61 to 0x29daa4c 2024-09-14T06:59:42.059 INFO:teuthology.orchestra.run.smithi007.stdout:870 write 0x9096273 thru 0x90a2d75 (0xcb03 bytes) 2024-09-14T06:59:42.063 INFO:teuthology.orchestra.run.smithi007.stdout:871 write 0x78fb4a0 thru 0x7906584 (0xb0e5 bytes) 2024-09-14T06:59:42.067 INFO:teuthology.orchestra.run.smithi007.stdout:872 read 0x81f744a thru 0x81ff5f6 (0x81ad bytes) 2024-09-14T06:59:42.067 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:42.067 INFO:teuthology.orchestra.run.smithi007.stdout:874 write 0x2ede114 thru 0x2ee2a24 (0x4911 bytes) 2024-09-14T06:59:42.079 INFO:teuthology.orchestra.run.smithi007.stdout:876 write 0xd28bd08 thru 0xd291e5f (0x6158 bytes) 2024-09-14T06:59:42.082 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:42.083 INFO:teuthology.orchestra.run.smithi007.stdout:878 clone 5 order 25 su 2097152 sc 4 2024-09-14T06:59:42.098 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:42.098 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:42.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:59:42.149 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:42.148+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:42.213 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:42.213+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:42.257 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:42.257 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:42.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:59:42.289 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T06:59:42.314 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:42.314+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:42.348 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone4 from 0xd291e60 (overlap 0x29daa4c) to 0x217bb70 2024-09-14T06:59:42.415 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:42.415 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:42.416 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:42.414+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:42.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:59:42.432 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-14T06:59:42.516 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:42.515+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:42.557 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:42.557 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:42.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:59:42.617 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:42.617+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:42.699 INFO:teuthology.orchestra.run.smithi007.stdout:879 read 0x64190ad thru 0x641de38 (0x4d8c bytes) 2024-09-14T06:59:42.700 INFO:teuthology.orchestra.run.smithi007.stdout:880 writesame 0x58756a8 thru 0x5876689 data_size 0x2(0xfe2 bytes) 2024-09-14T06:59:42.718 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:42.717+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:42.719 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:42.719 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:42.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:59:42.737 INFO:teuthology.orchestra.run.smithi007.stdout:881 writesame 0xd24f578 thru 0xd25255d data_size 0x1(0x2fe6 bytes) 2024-09-14T06:59:42.745 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:42.745 INFO:teuthology.orchestra.run.smithi007.stdout:887 read 0x1002fbe thru 0x100a0da (0x711d bytes) 2024-09-14T06:59:42.746 INFO:teuthology.orchestra.run.smithi007.stdout:889 write 0xcc79b6 thru 0xcd0956 (0x8fa1 bytes) 2024-09-14T06:59:42.783 INFO:teuthology.orchestra.run.smithi007.stdout:890 writesame 0x4048b4f thru 0x404da1e data_size 0x34(0x4ed0 bytes) 2024-09-14T06:59:42.785 INFO:teuthology.orchestra.run.smithi007.stdout:891 read 0x334a531 thru 0x334c384 (0x1e54 bytes) 2024-09-14T06:59:42.785 INFO:teuthology.orchestra.run.smithi007.stdout:892 read 0x1ee31c7 thru 0x1ef1c67 (0xeaa1 bytes) 2024-09-14T06:59:42.785 INFO:teuthology.orchestra.run.smithi007.stdout:893 read 0x83d8ad6 thru 0x83db4c5 (0x29f0 bytes) 2024-09-14T06:59:42.785 INFO:teuthology.orchestra.run.smithi007.stdout:894 punch from 0x5ed903c to 0x5edb0a2, (0x2066 bytes) 2024-09-14T06:59:42.785 INFO:teuthology.orchestra.run.smithi007.stdout:895 read 0x12dbf9b thru 0x12e61b8 (0xa21e bytes) 2024-09-14T06:59:42.786 INFO:teuthology.orchestra.run.smithi007.stdout:896 read 0x2f0f261 thru 0x2f1c8cd (0xd66d bytes) 2024-09-14T06:59:42.786 INFO:teuthology.orchestra.run.smithi007.stdout:898 read 0x9ce2105 thru 0x9ce70be (0x4fba bytes) 2024-09-14T06:59:42.786 INFO:teuthology.orchestra.run.smithi007.stdout:899 write 0x834bc8e thru 0x835bc60 (0xffd3 bytes) 2024-09-14T06:59:42.788 INFO:teuthology.orchestra.run.smithi007.stdout:901 write 0x5ab32d6 thru 0x5ab9e84 (0x6baf bytes) 2024-09-14T06:59:42.812 INFO:teuthology.orchestra.run.smithi007.stdout:902 writesame 0x29540dd thru 0x295a8c8 data_size 0x12(0x67ec bytes) 2024-09-14T06:59:42.814 INFO:teuthology.orchestra.run.smithi007.stdout:904 write 0x1cdf9f8 thru 0x1cec8bd (0xcec6 bytes) 2024-09-14T06:59:42.816 INFO:teuthology.orchestra.run.smithi007.stdout:905 read 0x4c54019 thru 0x4c5acae (0x6c96 bytes) 2024-09-14T06:59:42.817 INFO:teuthology.orchestra.run.smithi007.stdout:908 writesame 0xc3b2461 thru 0xc3bcb7f data_size 0xd(0xa71f bytes) 2024-09-14T06:59:42.819 INFO:teuthology.orchestra.run.smithi007.stdout:909 trunc from 0xd291e60 to 0xd3227ab 2024-09-14T06:59:42.819 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:42.818+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:42.824 INFO:teuthology.orchestra.run.smithi007.stdout:910 read 0x51865c7 thru 0x518e60d (0x8047 bytes) 2024-09-14T06:59:42.824 INFO:teuthology.orchestra.run.smithi007.stdout:911 writesame 0x49ff398 thru 0x4a0756f data_size 0xc(0x81d8 bytes) 2024-09-14T06:59:42.827 INFO:teuthology.orchestra.run.smithi007.stdout:914 read 0x4659832 thru 0x46607d0 (0x6f9f bytes) 2024-09-14T06:59:42.827 INFO:teuthology.orchestra.run.smithi007.stdout:915 read 0x2274dd thru 0x234db9 (0xd8dd bytes) 2024-09-14T06:59:42.827 INFO:teuthology.orchestra.run.smithi007.stdout:917 read 0x657fce9 thru 0x6585376 (0x568e bytes) 2024-09-14T06:59:42.827 INFO:teuthology.orchestra.run.smithi007.stdout:918 writesame 0x1fd9637 thru 0x1fe8735 data_size 0x2d(0xf0ff bytes) 2024-09-14T06:59:42.858 INFO:teuthology.orchestra.run.smithi007.stdout:919 writesame 0x2f7d thru 0xd3a7 data_size 0x1(0xa42b bytes) 2024-09-14T06:59:42.861 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:42.861 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:42.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:59:42.877 INFO:teuthology.orchestra.run.smithi007.stdout:920 read 0x6bb9f1e thru 0x6bbf48b (0x556e bytes) 2024-09-14T06:59:42.878 INFO:teuthology.orchestra.run.smithi007.stdout:921 write 0x118ef10 thru 0x119abda (0xbccb bytes) 2024-09-14T06:59:42.879 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:42.880 INFO:teuthology.orchestra.run.smithi007.stdout:924 punch from 0x78a26e5 to 0x78ae2c6, (0xbbe1 bytes) 2024-09-14T06:59:42.885 INFO:teuthology.orchestra.run.smithi007.stdout:925 write 0xd656299 thru 0xd657f17 (0x1c7f bytes) 2024-09-14T06:59:42.888 INFO:teuthology.orchestra.run.smithi007.stdout:926 writesame 0x66b3050 thru 0x66c1eb1 data_size 0x15(0xee62 bytes) 2024-09-14T06:59:42.890 INFO:teuthology.orchestra.run.smithi007.stdout:929 write 0x243c34e thru 0x24445cf (0x8282 bytes) 2024-09-14T06:59:42.892 INFO:teuthology.orchestra.run.smithi007.stdout:930 read 0x51ea335 thru 0x51f6af0 (0xc7bc bytes) 2024-09-14T06:59:42.893 INFO:teuthology.orchestra.run.smithi007.stdout:931 write 0xc10cf2e thru 0xc113831 (0x6904 bytes) 2024-09-14T06:59:42.897 INFO:teuthology.orchestra.run.smithi007.stdout:932 punch from 0x4df7e59 to 0x4e03ac2, (0xbc69 bytes) 2024-09-14T06:59:42.897 INFO:teuthology.orchestra.run.smithi007.stdout:933 punch from 0xa6115da to 0xa61f18c, (0xdbb2 bytes) 2024-09-14T06:59:42.897 INFO:teuthology.orchestra.run.smithi007.stdout:934 writesame 0x1256622 thru 0x126176b data_size 0xb(0xb14a bytes) 2024-09-14T06:59:42.911 INFO:teuthology.orchestra.run.smithi007.stdout:935 trunc from 0xd657f18 to 0x2879599 2024-09-14T06:59:42.919 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:42.919+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:42.935 INFO:teuthology.orchestra.run.smithi007.stdout:937 trunc from 0x2879599 to 0xc9bcc32 2024-09-14T06:59:42.940 INFO:teuthology.orchestra.run.smithi007.stdout:939 writesame 0x14e4e10 thru 0x14e8840 data_size 0x1(0x3a31 bytes) 2024-09-14T06:59:42.956 INFO:teuthology.orchestra.run.smithi007.stdout:940 trunc from 0xc9bcc32 to 0x58ac681 2024-09-14T06:59:42.962 INFO:teuthology.orchestra.run.smithi007.stdout:941 punch from 0x518059a to 0x5189fd0, (0x9a36 bytes) 2024-09-14T06:59:42.962 INFO:teuthology.orchestra.run.smithi007.stdout:943 read 0x2689b64 thru 0x26931c2 (0x965f bytes) 2024-09-14T06:59:42.975 INFO:teuthology.orchestra.run.smithi007.stdout:944 writesame 0xaab38e9 thru 0xaabee21 data_size 0x1(0xb539 bytes) 2024-09-14T06:59:42.986 INFO:teuthology.orchestra.run.smithi007.stdout:945 trunc from 0xaabee22 to 0x639e00 2024-09-14T06:59:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:43.004 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:43.012 INFO:teuthology.orchestra.run.smithi007.stdout:947 write 0xb670cda thru 0xb6730bf (0x23e6 bytes) 2024-09-14T06:59:43.016 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:59:43.016 INFO:teuthology.orchestra.run.smithi007.stdout:948 writesame 0x35b12d5 thru 0x35b66f7 data_size 0x11(0x5423 bytes) 2024-09-14T06:59:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:950 punch from 0x1231c36 to 0x123b0ae, (0x9478 bytes) 2024-09-14T06:59:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:951 read 0x2ea4a29 thru 0x2eaed19 (0xa2f1 bytes) 2024-09-14T06:59:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:43.018 INFO:teuthology.orchestra.run.smithi007.stdout:953 write 0x6beb8db thru 0x6bfad48 (0xf46e bytes) 2024-09-14T06:59:43.019 INFO:teuthology.orchestra.run.smithi007.stdout:954 read 0x4c8b32b thru 0x4c95e65 (0xab3b bytes) 2024-09-14T06:59:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:956 read 0x3670c92 thru 0x36734a4 (0x2813 bytes) 2024-09-14T06:59:43.020 INFO:teuthology.orchestra.run.smithi007.stdout:957 write 0x6637a95 thru 0x66464ce (0xea3a bytes) 2024-09-14T06:59:43.022 INFO:teuthology.orchestra.run.smithi007.stdout:958 write 0x52f2313 thru 0x5301809 (0xf4f7 bytes) 2024-09-14T06:59:43.023 INFO:teuthology.orchestra.run.smithi007.stdout:959 read 0x407c0c4 thru 0x408a10a (0xe047 bytes) 2024-09-14T06:59:43.025 INFO:teuthology.orchestra.run.smithi007.stdout:960 punch from 0x28019c0 to 0x2811305, (0xf945 bytes) 2024-09-14T06:59:43.025 INFO:teuthology.orchestra.run.smithi007.stdout:961 read 0x5c70016 thru 0x5c7be11 (0xbdfc bytes) 2024-09-14T06:59:43.025 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:43.025 INFO:teuthology.orchestra.run.smithi007.stdout:963 read 0x104eea8 thru 0x1051ddd (0x2f36 bytes) 2024-09-14T06:59:43.025 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:43.026 INFO:teuthology.orchestra.run.smithi007.stdout:966 punch from 0x88b13a2 to 0x88b5265, (0x3ec3 bytes) 2024-09-14T06:59:43.026 INFO:teuthology.orchestra.run.smithi007.stdout:970 write 0x3783a42 thru 0x378dbf6 (0xa1b5 bytes) 2024-09-14T06:59:43.026 INFO:teuthology.orchestra.run.smithi007.stdout:971 clone 6 order 21 su 8192 sc 14 2024-09-14T06:59:43.060 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:43.059+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:43.121 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:43.121 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:43.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:59:43.122 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:43.120+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:43.129 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:59:43.221 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:43.221+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:43.246 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:43.246 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:43.262 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:43.263 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:43.274 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:43.274 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:43.278 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:43.279 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:43.282 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:59:43.297 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:43.298 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:43.317 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:43.317 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:43.322 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:43.322+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:43.335 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:43.336 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:43.353 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone5 from 0xb6730c0 (overlap 0x639e00) to 0x1de299 2024-09-14T06:59:43.355 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:43.378 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:43.378 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:43.421 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-14T06:59:43.423 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:43.423+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1839 3.19 3:9a80ba12:::rbd_data.114fe955cd3f.0000000000000004:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.800958+0000", 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.6321227350000003, 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.028916818, 2024-09-14T06:59:43.435 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.436 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.436 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.436 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.436 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.436 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.436 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1839 2024-09-14T06:59:43.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.436 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.800958+0000", 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.800958+0000", 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.800960+0000", 2024-09-14T06:59:43.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.934e-06 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.800972+0000", 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2659e-05 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.801000+0000", 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7399999999999999e-05 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.801013+0000", 2024-09-14T06:59:43.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.365e-05 2024-09-14T06:59:43.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.801153+0000", 2024-09-14T06:59:43.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013930299999999999 2024-09-14T06:59:43.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.801299+0000", 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014683299999999999 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.806031+0000", 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.004731458 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829785+0000", 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.023754016999999999 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829804+0000", 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9471999999999999e-05 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829839+0000", 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4147999999999999e-05 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829874+0000", 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5944e-05 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.442 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop_reply(client.4431.0:1839 3.19 e30/24 ondisk, result = 0)", 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.821228+0000", 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.6118522049999999, 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0085620130000000003, 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821228+0000", 2024-09-14T06:59:43.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821228+0000", 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821229+0000", 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0100000000000004e-07 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821233+0000", 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1219999999999997e-06 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821244+0000", 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0977e-05 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.821268+0000", 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4328000000000002e-05 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829771+0000", 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0085023330000000008 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829779+0000", 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.49e-06 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.829790+0000", 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0961999999999999e-05 2024-09-14T06:59:43.446 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1844 3.19 3:9a80ba12:::rbd_data.114fe955cd3f.0000000000000004:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837247+0000", 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.5958336959999997, 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002587275, 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.447 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1844 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837247+0000", 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837247+0000", 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837249+0000", 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.74e-06 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837261+0000", 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1263e-05 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837293+0000", 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2363000000000002e-05 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837317+0000", 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3540000000000002e-05 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837608+0000", 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029157199999999998 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9833999999999994e-05 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837843+0000", 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014475999999999999 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838547+0000", 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00070457199999999999 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.839772+0000", 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0012246329999999999 2024-09-14T06:59:43.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.839813+0000", 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1182e-05 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.839834+0000", 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0815999999999999e-05 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1876 3.19 3:9a80ba12:::rbd_data.114fe955cd3f.0000000000000004:head [stat,zero 14776037~952603] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863737+0000", 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.5693436160000003, 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0026618179999999998, 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.453 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1876 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863737+0000", 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863737+0000", 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863739+0000", 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4870000000000001e-06 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863750+0000", 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.116e-05 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863786+0000", 2024-09-14T06:59:43.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5985999999999997e-05 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863819+0000", 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2409000000000001e-05 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863976+0000", 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015743800000000001 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864109+0000", 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013315700000000001 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866399+0000", 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022891809999999999 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:1939 3.9 e31/24 3:93377bc8:::rbd_header.114fe955cd3f:head v 31'100, mlcod=30'98)", 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.279581+0000", 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.1534992239999999, 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044217299999999996, 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279581+0000", 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279581+0000", 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279582+0000", 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279679+0000", 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7362999999999996e-05 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279688+0000", 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4260000000000003e-06 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279696+0000", 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6270000000000006e-06 2024-09-14T06:59:43.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279831+0000", 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013447700000000001 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.279851+0000", 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0254e-05 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.283955+0000", 2024-09-14T06:59:43.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0041038940000000003 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.284003+0000", 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8049e-05 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2002 3.6 3:61705fc6:::rbd_header.114fe2819363:head [call rbd.set_stripe_unit_count in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.1092986409999996, 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031864329999999998, 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2002 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323782+0000", 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6899999999999997e-07 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323802+0000", 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9641999999999999e-05 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323813+0000", 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1263999999999999e-05 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323818+0000", 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5390000000000001e-06 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323858+0000", 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0556000000000003e-05 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.323904+0000", 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5383000000000002e-05 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.324099+0000", 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000195157 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.326319+0000", 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022197139999999998 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.326908+0000", 2024-09-14T06:59:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058982900000000005 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.326942+0000", 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3275999999999998e-05 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.326968+0000", 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6503999999999999e-05 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:2017 3.6 3:61705fc6:::rbd_header.114fe2819363:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.337448+0000", 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.0956326929999998, 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024757949999999998, 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 2017 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337448+0000", 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337448+0000", 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337449+0000", 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999995e-07 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337459+0000", 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0712e-05 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337472+0000", 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2267e-05 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337479+0000", 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.294e-06 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337516+0000", 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7184999999999997e-05 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337565+0000", 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8813999999999997e-05 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.337706+0000", 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000140636 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.338110+0000", 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00040478299999999998 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.339872+0000", 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001761486 2024-09-14T06:59:43.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.339902+0000", 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.987e-05 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.339924+0000", 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1803000000000001e-05 2024-09-14T06:59:43.474 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3149 3.8 3:14b258c9:::rbd_data.114fe2819363.000000000000004c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.495211+0000", 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.9378698229999998, 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0026394470000000001, 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.475 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.476 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3149 2024-09-14T06:59:43.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.476 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495211+0000", 2024-09-14T06:59:43.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495211+0000", 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495211+0000", 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5299999999999999e-07 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495225+0000", 2024-09-14T06:59:43.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3936e-05 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495232+0000", 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7700000000000004e-06 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495237+0000", 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3800000000000002e-06 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495284+0000", 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6625999999999997e-05 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495424+0000", 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014056499999999999 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:43.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.495624+0000", 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019920800000000001 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.496566+0000", 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00094286299999999995 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.497758+0000", 2024-09-14T06:59:43.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001191576 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.497813+0000", 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5461999999999999e-05 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.497850+0000", 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6607999999999999e-05 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.481 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3390 3.8 3:14b258c9:::rbd_data.114fe2819363.000000000000004c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632022+0000", 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.8010581439999997, 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066451030000000003, 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.482 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3390 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632022+0000", 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632022+0000", 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632023+0000", 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2600000000000002e-07 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632026+0000", 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9440000000000001e-06 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632036+0000", 2024-09-14T06:59:43.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8840000000000006e-06 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632042+0000", 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2160000000000003e-06 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632740+0000", 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069763699999999998 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632824+0000", 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4677000000000003e-05 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632931+0000", 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107129 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.635486+0000", 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025543609999999998 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.638574+0000", 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030884760000000002 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.638626+0000", 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1323000000000003e-05 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.638667+0000", 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1829999999999998e-05 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3396 3.19 3:9bfba276:::rbd_data.114fe2819363.000000000000005f:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.633909+0000", 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.7991709130000002, 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025159729999999999, 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3396 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633909+0000", 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633909+0000", 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633910+0000", 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8100000000000001e-07 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633915+0000", 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4530000000000004e-06 2024-09-14T06:59:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633929+0000", 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3759e-05 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633937+0000", 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1219999999999995e-06 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633990+0000", 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2842e-05 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.634033+0000", 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3596000000000001e-05 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.634087+0000", 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4109e-05 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.635158+0000", 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001070688 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636355+0000", 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00119651 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636391+0000", 2024-09-14T06:59:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6294000000000002e-05 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636425+0000", 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4619000000000002e-05 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3406 3.1b 3:d8f35a5b:::rbd_data.114fe2819363.0000000000000074:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.636669+0000", 2024-09-14T06:59:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.796411848, 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.004578922, 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3406 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636669+0000", 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636669+0000", 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636669+0000", 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3699999999999999e-07 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636675+0000", 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1510000000000003e-06 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636684+0000", 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3950000000000007e-06 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636689+0000", 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9540000000000003e-06 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636733+0000", 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.371e-05 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636781+0000", 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8025000000000002e-05 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636856+0000", 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5860999999999998e-05 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.638017+0000", 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00116081 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.641150+0000", 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031323100000000001 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.641205+0000", 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4928e-05 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.641247+0000", 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2941000000000001e-05 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3408 3.6 3:66853b88:::rbd_data.114fe2819363.000000000000007c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.637467+0000", 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.795613736, 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040210239999999998, 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3408 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637467+0000", 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637467+0000", 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637467+0000", 2024-09-14T06:59:43.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7199999999999998e-07 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637471+0000", 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6449999999999998e-06 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637480+0000", 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637485+0000", 2024-09-14T06:59:43.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5700000000000003e-06 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637527+0000", 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2307e-05 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637573+0000", 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6202000000000003e-05 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:43.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637631+0000", 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8226000000000001e-05 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.641425+0000", 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037939720000000001 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.641437+0000", 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1637999999999999e-05 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.641463+0000", 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6395000000000001e-05 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.641488+0000", 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4206000000000002e-05 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3409 3.9 e32/24 3:95f5acae:::rbd_data.114fe2819363.0000000000000180:head v 32'113, mlcod=32'111)", 2024-09-14T06:59:43.506 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.637893+0000", 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.7951877120000002, 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0035664429999999999, 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637893+0000", 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637893+0000", 2024-09-14T06:59:43.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637894+0000", 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.108e-06 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637899+0000", 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2320000000000001e-06 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637912+0000", 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2806000000000001e-05 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637919+0000", 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1890000000000001e-06 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637955+0000", 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5720999999999999e-05 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.637966+0000", 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1082e-05 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.641421+0000", 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003454857 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.641459+0000", 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8448000000000002e-05 2024-09-14T06:59:43.510 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3446 3.19 3:9b33cc4e:::rbd_data.114fe2819363.000000000000016e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.679570+0000", 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.7535108880000001, 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0026198419999999998, 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.511 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3446 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.679570+0000", 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.679570+0000", 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.679570+0000", 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0100000000000004e-07 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.679574+0000", 2024-09-14T06:59:43.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0620000000000002e-06 2024-09-14T06:59:43.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.679586+0000", 2024-09-14T06:59:43.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1121000000000001e-05 2024-09-14T06:59:43.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.679590+0000", 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9799999999999998e-06 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.679639+0000", 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8256e-05 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.679724+0000", 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5360999999999998e-05 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.680164+0000", 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043947099999999998 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.682100+0000", 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019359830000000001 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.682131+0000", 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1062999999999997e-05 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.682162+0000", 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1115999999999998e-05 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.682189+0000", 2024-09-14T06:59:43.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7628e-05 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3450 3.6 3:61705fc6:::rbd_header.114fe2819363:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.697154+0000", 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.7359268569999999, 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003876493, 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.518 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3450 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.697154+0000", 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.697154+0000", 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.697154+0000", 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.697159+0000", 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6090000000000003e-06 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.697170+0000", 2024-09-14T06:59:43.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.697175+0000", 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2739999999999998e-06 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.697232+0000", 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6974e-05 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.697326+0000", 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3358999999999998e-05 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.697553+0000", 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000227655 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.700810+0000", 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032569819999999998 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.700963+0000", 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015233600000000001 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.700997+0000", 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4866000000000001e-05 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.701030+0000", 2024-09-14T06:59:43.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2558000000000002e-05 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3503 3.6 3:61705fc6:::rbd_header.114fe2819363:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.771396+0000", 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.66168425, 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0037288519999999999, 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.524 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3503 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.771396+0000", 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.771396+0000", 2024-09-14T06:59:43.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.771397+0000", 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8400000000000003e-07 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.771401+0000", 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7349999999999998e-06 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.771411+0000", 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0778000000000001e-05 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.771416+0000", 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9189999999999998e-06 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.771465+0000", 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8952999999999997e-05 2024-09-14T06:59:43.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.771541+0000", 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5467000000000001e-05 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.771737+0000", 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019660500000000001 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.772484+0000", 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00074614400000000002 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.775057+0000", 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025731069999999998 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.775087+0000", 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0156000000000001e-05 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.775125+0000", 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3552 3.2 3:407ac397:::rbd_data.114fe2819363.00000000000000a5:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.819585+0000", 2024-09-14T06:59:43.530 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.6134949450000002, 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.018885210999999999, 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3552 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.819585+0000", 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.819585+0000", 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.819586+0000", 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9100000000000003e-07 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.819590+0000", 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.681e-06 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.819600+0000", 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0291e-05 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.819605+0000", 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7620000000000003e-06 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.819667+0000", 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2018000000000002e-05 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.819743+0000", 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6266000000000005e-05 2024-09-14T06:59:43.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 2", 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.822835+0000", 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030920850000000001 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.838391+0000", 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.015555309 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.838403+0000", 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2731999999999999e-05 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.838438+0000", 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4530999999999999e-05 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.838471+0000", 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2744999999999999e-05 2024-09-14T06:59:43.536 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop_reply(client.4431.0:3552 3.2 e32/24 ondisk, result = 0)", 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.832887+0000", 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.6001932089999999, 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0055097660000000001, 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:43.537 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.832887+0000", 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.832887+0000", 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.832888+0000", 2024-09-14T06:59:43.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1999999999999998e-07 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.832892+0000", 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7469999999999999e-06 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.832904+0000", 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2472e-05 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.832912+0000", 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3529999999999997e-06 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.838379+0000", 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.005467497 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.838386+0000", 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2479999999999999e-06 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.838397+0000", 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0529e-05 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3554 3.6 3:61705fc6:::rbd_header.114fe2819363:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.841946+0000", 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.5911346129999999, 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033297069999999999, 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.541 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3554 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.841946+0000", 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.841946+0000", 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.841947+0000", 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6750000000000001e-06 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.841952+0000", 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9099999999999996e-06 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.841963+0000", 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0961000000000001e-05 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.841969+0000", 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7810000000000002e-06 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.842042+0000", 2024-09-14T06:59:43.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2410999999999995e-05 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.842102+0000", 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0117000000000001e-05 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.842230+0000", 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012822699999999999 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.843068+0000", 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00083817099999999997 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.845205+0000", 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021369100000000001 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.845237+0000", 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2484000000000003e-05 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.845275+0000", 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8059999999999998e-05 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:3583 3.6 3:60baf965:::rbd_data.114fe2819363.000000000000016c:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.866184+0000", 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 2.5668960150000002, 2024-09-14T06:59:43.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0035630969999999999, 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 3583 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.866184+0000", 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.866184+0000", 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.866186+0000", 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1799999999999999e-06 2024-09-14T06:59:43.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.866191+0000", 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.866202+0000", 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.113e-05 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.866207+0000", 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3770000000000004e-06 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.866251+0000", 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4016e-05 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.866345+0000", 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3745000000000005e-05 2024-09-14T06:59:43.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 7", 2024-09-14T06:59:43.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.866921+0000", 2024-09-14T06:59:43.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00057660199999999995 2024-09-14T06:59:43.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:43.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.869657+0000", 2024-09-14T06:59:43.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002735365 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.869687+0000", 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0320000000000001e-05 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.869720+0000", 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2354000000000002e-05 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.869747+0000", 2024-09-14T06:59:43.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7948000000000001e-05 2024-09-14T06:59:43.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.554 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.554 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.554 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:43.555 INFO:teuthology.orchestra.run.smithi007.stdout:972 write 0x6a9c034 thru 0x6aab69b (0xf668 bytes) 2024-09-14T06:59:43.556 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:43.554+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:43.556 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:59:43.611 INFO:teuthology.orchestra.run.smithi007.stdout:973 read 0xe3a26f thru 0xe406d8 (0x646a bytes) 2024-09-14T06:59:43.612 INFO:teuthology.orchestra.run.smithi007.stdout:975 read 0x459b2c0 thru 0x459db1a (0x285b bytes) 2024-09-14T06:59:43.619 INFO:teuthology.orchestra.run.smithi007.stdout:977 write 0xcc3e30a thru 0xcc43b31 (0x5828 bytes) 2024-09-14T06:59:43.631 INFO:teuthology.orchestra.run.smithi007.stdout:978 write 0xa89e17f thru 0xa89f1d6 (0x1058 bytes) 2024-09-14T06:59:43.637 INFO:teuthology.orchestra.run.smithi007.stdout:979 read 0x10f74ad thru 0x1104b12 (0xd666 bytes) 2024-09-14T06:59:43.638 INFO:teuthology.orchestra.run.smithi007.stdout:980 write 0x873c730 thru 0x8740a3e (0x430f bytes) 2024-09-14T06:59:43.656 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:43.655+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:43.664 INFO:teuthology.orchestra.run.smithi007.stdout:981 write 0x9ed38ee thru 0x9ed8567 (0x4c7a bytes) 2024-09-14T06:59:43.709 INFO:teuthology.orchestra.run.smithi007.stdout:982 write 0x2393e99 thru 0x239407a (0x1e2 bytes) 2024-09-14T06:59:43.738 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:43.738 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:43.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.738 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8943 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.738 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722806+0000", 2024-09-14T06:59:43.738 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013203408999999999, 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013226296, 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8943 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722806+0000", 2024-09-14T06:59:43.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722806+0000", 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722806+0000", 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.29e-07 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722808+0000", 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4139999999999999e-06 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722812+0000", 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.126e-06 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722815+0000", 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8509999999999999e-06 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.741 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8944 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722818+0000", 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013191642999999999, 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.01327331, 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.742 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8944 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722818+0000", 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722818+0000", 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722818+0000", 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722819+0000", 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.057e-06 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722822+0000", 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1300000000000001e-06 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722825+0000", 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5339999999999998e-06 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.745 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8945 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722828+0000", 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013181761, 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013310371, 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8945 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722828+0000", 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722828+0000", 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722828+0000", 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.34e-07 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722829+0000", 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0860000000000001e-06 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722832+0000", 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722835+0000", 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4879999999999999e-06 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8946 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722839+0000", 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013170671, 2024-09-14T06:59:43.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013346594, 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8946 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722839+0000", 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722839+0000", 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722839+0000", 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722840+0000", 2024-09-14T06:59:43.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7699999999999992e-07 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722843+0000", 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5999999999999998e-06 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722849+0000", 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0950000000000003e-06 2024-09-14T06:59:43.752 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8947 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722854+0000", 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013155159, 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013372021, 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.753 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8947 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722854+0000", 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722854+0000", 2024-09-14T06:59:43.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722855+0000", 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722857+0000", 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0719999999999998e-06 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722863+0000", 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722867+0000", 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5940000000000004e-06 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8948 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.756 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722874+0000", 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013135343000000001, 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013379975000000001, 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8948 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722874+0000", 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722874+0000", 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722874+0000", 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6399999999999998e-07 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722876+0000", 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1059999999999998e-06 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722883+0000", 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1999999999999999e-06 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722886+0000", 2024-09-14T06:59:43.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7120000000000002e-06 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8949 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722890+0000", 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013118955, 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013386466, 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.760 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8949 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722890+0000", 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722890+0000", 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722891+0000", 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3000000000000002e-07 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722892+0000", 2024-09-14T06:59:43.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2049999999999999e-06 2024-09-14T06:59:43.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722895+0000", 2024-09-14T06:59:43.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2959999999999999e-06 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722898+0000", 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4040000000000002e-06 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8950 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722901+0000", 2024-09-14T06:59:43.765 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013108783000000001, 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013399405, 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8950 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722901+0000", 2024-09-14T06:59:43.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722901+0000", 2024-09-14T06:59:43.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722901+0000", 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722902+0000", 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.122e-06 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722905+0000", 2024-09-14T06:59:43.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:59:43.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722908+0000", 2024-09-14T06:59:43.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7470000000000001e-06 2024-09-14T06:59:43.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.769 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8951 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722935+0000", 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013074572, 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013388056000000001, 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.770 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8951 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722935+0000", 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722935+0000", 2024-09-14T06:59:43.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722935+0000", 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5700000000000002e-07 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722941+0000", 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722948+0000", 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1210000000000003e-06 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722953+0000", 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.972e-06 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.773 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8952 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722960+0000", 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013049793000000001, 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013387896, 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8952 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722960+0000", 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722960+0000", 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722960+0000", 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1899999999999998e-07 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722962+0000", 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9130000000000001e-06 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722968+0000", 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.032e-06 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722973+0000", 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6500000000000004e-06 2024-09-14T06:59:43.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8953 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722979+0000", 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013030412999999999, 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013391769, 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.777 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8953 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722979+0000", 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722979+0000", 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722979+0000", 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722981+0000", 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.711e-06 2024-09-14T06:59:43.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722987+0000", 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8889999999999999e-06 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722992+0000", 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7530000000000001e-06 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8954 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722997+0000", 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013011841999999999, 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013396028000000001, 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8954 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.781 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722997+0000", 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722997+0000", 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722998+0000", 2024-09-14T06:59:43.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-07 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.723000+0000", 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8530000000000001e-06 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.723007+0000", 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2960000000000001e-06 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.723013+0000", 2024-09-14T06:59:43.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6729999999999997e-06 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8955 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.723019+0000", 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.012990602, 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013399901000000001, 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.784 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8955 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.723019+0000", 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.723019+0000", 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.723019+0000", 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2499999999999995e-07 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.723021+0000", 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9209999999999999e-06 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.723027+0000", 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0460000000000002e-06 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.723032+0000", 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7840000000000003e-06 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8925 3.1f 3:f80707c6:::rbd_data.114f7dc3575d.0000000000000002:8 [read 11173888~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722458+0000", 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013551453999999999, 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014009479, 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8925 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722458+0000", 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722458+0000", 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722458+0000", 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2099999999999998e-07 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722460+0000", 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.623e-06 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722466+0000", 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7270000000000004e-06 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722471+0000", 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9309999999999996e-06 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.736359+0000", 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.013887488999999999 2024-09-14T06:59:43.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.736436+0000", 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6823000000000006e-05 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8926 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.792 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722477+0000", 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013532512, 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014046295, 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8926 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722477+0000", 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722477+0000", 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722477+0000", 2024-09-14T06:59:43.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722479+0000", 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1270000000000001e-06 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722485+0000", 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8699999999999997e-06 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722490+0000", 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5809999999999998e-06 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8927 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722495+0000", 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013514165999999999, 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.01407541, 2024-09-14T06:59:43.796 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8927 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722495+0000", 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722495+0000", 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722496+0000", 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2499999999999995e-07 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722502+0000", 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4779999999999996e-06 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722508+0000", 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0510000000000002e-06 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722513+0000", 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8160000000000001e-06 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.799 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8928 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722542+0000", 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013467435999999999, 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0140778, 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8928 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722542+0000", 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722542+0000", 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722542+0000", 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6899999999999997e-07 2024-09-14T06:59:43.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722544+0000", 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.029e-06 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722551+0000", 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3430000000000001e-06 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722556+0000", 2024-09-14T06:59:43.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0370000000000002e-06 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8929 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722562+0000", 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013447008, 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014096900000000001, 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.803 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8929 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722562+0000", 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722562+0000", 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722563+0000", 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0299999999999999e-07 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722565+0000", 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.889e-06 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722571+0000", 2024-09-14T06:59:43.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2620000000000004e-06 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722576+0000", 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7879999999999997e-06 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8930 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722581+0000", 2024-09-14T06:59:43.806 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013427926999999999, 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014103183, 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8930 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722581+0000", 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722581+0000", 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722582+0000", 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1899999999999998e-07 2024-09-14T06:59:43.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722583+0000", 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722589+0000", 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8340000000000004e-06 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722594+0000", 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5310000000000003e-06 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8931 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722599+0000", 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013410155999999999, 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014108552999999999, 2024-09-14T06:59:43.810 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8931 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722599+0000", 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722599+0000", 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722599+0000", 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:59:43.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722601+0000", 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6130000000000001e-06 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722607+0000", 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7379999999999999e-06 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722611+0000", 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.549e-06 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8932 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722617+0000", 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013392677, 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014113868, 2024-09-14T06:59:43.814 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8932 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722617+0000", 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722617+0000", 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722617+0000", 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0199999999999998e-07 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722619+0000", 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.637e-06 2024-09-14T06:59:43.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722624+0000", 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722629+0000", 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4959999999999998e-06 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8933 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722634+0000", 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013375075, 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014119216, 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8933 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722634+0000", 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722634+0000", 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722635+0000", 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:59:43.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722636+0000", 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722642+0000", 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8309999999999997e-06 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722647+0000", 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5689999999999998e-06 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8934 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722652+0000", 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013357325999999999, 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014124207999999999, 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.821 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8934 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722652+0000", 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722652+0000", 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722652+0000", 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.27e-07 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722654+0000", 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.81e-06 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722660+0000", 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8510000000000004e-06 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722664+0000", 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4900000000000002e-06 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.824 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8935 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722670+0000", 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013339268, 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014139039000000001, 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8935 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722670+0000", 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722670+0000", 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722670+0000", 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722672+0000", 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.818e-06 2024-09-14T06:59:43.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722678+0000", 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722683+0000", 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4689999999999999e-06 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8936 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722688+0000", 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013321357000000001, 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014155657, 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.829 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8936 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722688+0000", 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722688+0000", 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722689+0000", 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.51e-07 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722690+0000", 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6619999999999999e-06 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722696+0000", 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8690000000000001e-06 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722704+0000", 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5859999999999999e-06 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8937 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722730+0000", 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013279391999999999, 2024-09-14T06:59:43.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014160212, 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8937 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722730+0000", 2024-09-14T06:59:43.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722730+0000", 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722730+0000", 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.77e-07 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722732+0000", 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6330000000000001e-06 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722736+0000", 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4129999999999999e-06 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722739+0000", 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0510000000000001e-06 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8938 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722744+0000", 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013265677999999999, 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014188686000000001, 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8938 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722744+0000", 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722744+0000", 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722744+0000", 2024-09-14T06:59:43.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9700000000000003e-07 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722745+0000", 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.062e-06 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722748+0000", 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2119999999999999e-06 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722751+0000", 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0259999999999999e-06 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8939 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722754+0000", 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013254976999999999, 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014225574, 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.839 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8939 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722754+0000", 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722754+0000", 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722755+0000", 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.17e-07 2024-09-14T06:59:43.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722758+0000", 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8229999999999997e-06 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722762+0000", 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5659999999999998e-06 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722765+0000", 2024-09-14T06:59:43.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.604e-06 2024-09-14T06:59:43.843 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.843 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.843 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.843 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8940 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.843 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722768+0000", 2024-09-14T06:59:43.843 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013241418, 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014264565999999999, 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8940 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722768+0000", 2024-09-14T06:59:43.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722768+0000", 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722768+0000", 2024-09-14T06:59:43.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722769+0000", 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1039999999999999e-06 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722772+0000", 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0920000000000002e-06 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722775+0000", 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.683e-06 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8941 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722778+0000", 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013231402, 2024-09-14T06:59:43.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.01429772, 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8941 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722778+0000", 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722778+0000", 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722778+0000", 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722779+0000", 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0279999999999999e-06 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722782+0000", 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9909999999999998e-06 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722785+0000", 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9170000000000002e-06 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.850 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8942 3.1f 3.63e0e01f (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.722788+0000", 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.013221200000000001, 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.014315199000000001, 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8942 2024-09-14T06:59:43.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722788+0000", 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722788+0000", 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:43.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722788+0000", 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722789+0000", 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0860000000000001e-06 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722793+0000", 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.151e-06 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:43.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:43.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.722802+0000", 2024-09-14T06:59:43.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3999999999999998e-06 2024-09-14T06:59:43.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.854 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:43.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:43.854 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-14T06:59:43.854 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 31 2024-09-14T06:59:43.854 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:43.856 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:59:43.856 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:43.855+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:984 punch from 0xc279e10 to 0xc289cdf, (0xfecf bytes) 2024-09-14T06:59:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:986 read 0x39622fc thru 0x3967165 (0x4e6a bytes) 2024-09-14T06:59:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:988 read 0xa69d095 thru 0xa6a340a (0x6376 bytes) 2024-09-14T06:59:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:989 write 0xd2ccd55 thru 0xd2d0ec7 (0x4173 bytes) 2024-09-14T06:59:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:990 writesame 0x84fda10 thru 0x8509635 data_size 0x1(0xbc26 bytes) 2024-09-14T06:59:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:997 write 0xc10a1cc thru 0xc10d15e (0x2f93 bytes) 2024-09-14T06:59:43.858 INFO:teuthology.orchestra.run.smithi007.stdout:998 read 0x1511d8b thru 0x151bbcb (0x9e41 bytes) 2024-09-14T06:59:43.858 INFO:teuthology.orchestra.run.smithi007.stdout:999 punch from 0x9ea5553 to 0x9eb147e, (0xbf2b bytes) 2024-09-14T06:59:43.858 INFO:teuthology.orchestra.run.smithi007.stdout:1000 write 0x7b3586c thru 0x7b3cd4f (0x74e4 bytes) 2024-09-14T06:59:43.859 INFO:teuthology.orchestra.run.smithi007.stdout:1003 trunc from 0xd2d0ec8 to 0x432b7f7 2024-09-14T06:59:43.956 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:43.955+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:44.049 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:44.050 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:44.050 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:44.050 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:44.050 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:44.057 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:44.056+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:44.059 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:59:44.070 INFO:teuthology.orchestra.run.smithi007.stdout:1005 write 0x6758bf5 thru 0x6766700 (0xdb0c bytes) 2024-09-14T06:59:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:1007 punch from 0x3b80737 to 0x3b8f08c, (0xe955 bytes) 2024-09-14T06:59:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:44.079 INFO:teuthology.orchestra.run.smithi007.stdout:1010 read 0x1c6bd5e thru 0x1c73350 (0x75f3 bytes) 2024-09-14T06:59:44.087 INFO:teuthology.orchestra.run.smithi007.stdout:1011 write 0x78b98a1 thru 0x78c5d4b (0xc4ab bytes) 2024-09-14T06:59:44.095 INFO:teuthology.orchestra.run.smithi007.stdout:1013 punch from 0x66f673c to 0x6700854, (0xa118 bytes) 2024-09-14T06:59:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:1016 trunc from 0x78c5d4c to 0x5ec4e0f 2024-09-14T06:59:44.109 INFO:teuthology.orchestra.run.smithi007.stdout:1017 read 0x1eefd13 thru 0x1efdc6a (0xdf58 bytes) 2024-09-14T06:59:44.110 INFO:teuthology.orchestra.run.smithi007.stdout:1018 flatten 2024-09-14T06:59:44.158 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:44.157+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:44.205 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:44.205 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15433 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 5799936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.148233+0000", 2024-09-14T06:59:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.053808350999999997, 2024-09-14T06:59:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040312036000000002, 2024-09-14T06:59:44.206 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15433 2024-09-14T06:59:44.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148233+0000", 2024-09-14T06:59:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148233+0000", 2024-09-14T06:59:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148234+0000", 2024-09-14T06:59:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8400000000000005e-07 2024-09-14T06:59:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148236+0000", 2024-09-14T06:59:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.385e-06 2024-09-14T06:59:44.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148242+0000", 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3879999999999997e-06 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148248+0000", 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7289999999999997e-06 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.188400+0000", 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040151961999999999 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.188473+0000", 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3353000000000001e-05 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.188545+0000", 2024-09-14T06:59:44.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1735000000000004e-05 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15434 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 5914624~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.148256+0000", 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.053785504999999997, 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040502693999999999, 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.213 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15434 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148256+0000", 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148256+0000", 2024-09-14T06:59:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148256+0000", 2024-09-14T06:59:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7100000000000002e-07 2024-09-14T06:59:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148258+0000", 2024-09-14T06:59:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0719999999999998e-06 2024-09-14T06:59:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148265+0000", 2024-09-14T06:59:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0390000000000001e-06 2024-09-14T06:59:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148269+0000", 2024-09-14T06:59:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.51e-06 2024-09-14T06:59:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.188589+0000", 2024-09-14T06:59:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040319753999999999 2024-09-14T06:59:44.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.188679+0000", 2024-09-14T06:59:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9821000000000006e-05 2024-09-14T06:59:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.188759+0000", 2024-09-14T06:59:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0026999999999998e-05 2024-09-14T06:59:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15435 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 6029312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.219 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.148275+0000", 2024-09-14T06:59:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.053766745999999997, 2024-09-14T06:59:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040717974999999997, 2024-09-14T06:59:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.220 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15435 2024-09-14T06:59:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148275+0000", 2024-09-14T06:59:44.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.148275+0000", 2024-09-14T06:59:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.149839+0000", 2024-09-14T06:59:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0015640840000000001 2024-09-14T06:59:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.149844+0000", 2024-09-14T06:59:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2909999999999998e-06 2024-09-14T06:59:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.149855+0000", 2024-09-14T06:59:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0738e-05 2024-09-14T06:59:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.149862+0000", 2024-09-14T06:59:44.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6410000000000002e-06 2024-09-14T06:59:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.188812+0000", 2024-09-14T06:59:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038949696999999998 2024-09-14T06:59:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.188901+0000", 2024-09-14T06:59:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8449000000000006e-05 2024-09-14T06:59:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.188993+0000", 2024-09-14T06:59:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2075000000000002e-05 2024-09-14T06:59:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15443 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 6963200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150054+0000", 2024-09-14T06:59:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051987736, 2024-09-14T06:59:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040424098999999998, 2024-09-14T06:59:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.227 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15443 2024-09-14T06:59:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150054+0000", 2024-09-14T06:59:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150054+0000", 2024-09-14T06:59:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150054+0000", 2024-09-14T06:59:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6199999999999999e-07 2024-09-14T06:59:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150056+0000", 2024-09-14T06:59:44.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9640000000000002e-06 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150059+0000", 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3859999999999999e-06 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150062+0000", 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190350+0000", 2024-09-14T06:59:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040288526999999998 2024-09-14T06:59:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190417+0000", 2024-09-14T06:59:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6276e-05 2024-09-14T06:59:44.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190478+0000", 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1087999999999995e-05 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15444 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150065+0000", 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051976119000000001, 2024-09-14T06:59:44.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040600497999999999, 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15444 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150065+0000", 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150065+0000", 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150066+0000", 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150074+0000", 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0809999999999994e-06 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150077+0000", 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5810000000000001e-06 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150080+0000", 2024-09-14T06:59:44.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190517+0000", 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040436567 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190595+0000", 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8102999999999994e-05 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190666+0000", 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0735000000000006e-05 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15445 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 7192576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150084+0000", 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051957456999999999, 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040753268000000002, 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.238 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15445 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150084+0000", 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150084+0000", 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150084+0000", 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.23e-07 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150085+0000", 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.254e-06 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150089+0000", 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.19e-06 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150091+0000", 2024-09-14T06:59:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6690000000000002e-06 2024-09-14T06:59:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190696+0000", 2024-09-14T06:59:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040604625999999998 2024-09-14T06:59:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190768+0000", 2024-09-14T06:59:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2235000000000002e-05 2024-09-14T06:59:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190837+0000", 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9071000000000003e-05 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15446 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 7307264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150095+0000", 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051946808999999997, 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040934434999999998, 2024-09-14T06:59:44.244 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15446 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150095+0000", 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150095+0000", 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150095+0000", 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150096+0000", 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.361e-06 2024-09-14T06:59:44.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150099+0000", 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.247e-06 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150102+0000", 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6740000000000001e-06 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190877+0000", 2024-09-14T06:59:44.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040775155 2024-09-14T06:59:44.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.190953+0000", 2024-09-14T06:59:44.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5949000000000001e-05 2024-09-14T06:59:44.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191029+0000", 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5839000000000004e-05 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15447 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 7421952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150105+0000", 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051936098999999999, 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041090950000000001, 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.249 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15447 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150105+0000", 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150105+0000", 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150106+0000", 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5400000000000002e-07 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150107+0000", 2024-09-14T06:59:44.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1990000000000001e-06 2024-09-14T06:59:44.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150110+0000", 2024-09-14T06:59:44.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.19e-06 2024-09-14T06:59:44.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150113+0000", 2024-09-14T06:59:44.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3670000000000001e-06 2024-09-14T06:59:44.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191059+0000", 2024-09-14T06:59:44.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040945273999999997 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191132+0000", 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3586999999999999e-05 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191196+0000", 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3979000000000005e-05 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15448 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 7536640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150132+0000", 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051909258, 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041250097999999999, 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15448 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.255 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150132+0000", 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150132+0000", 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150133+0000", 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150134+0000", 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.578e-06 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150138+0000", 2024-09-14T06:59:44.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3560000000000003e-06 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150141+0000", 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8760000000000001e-06 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191242+0000", 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041100796000000002 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191316+0000", 2024-09-14T06:59:44.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3616999999999997e-05 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191382+0000", 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6493000000000002e-05 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15449 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 7651328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.259 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150146+0000", 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051895801999999998, 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041408966999999998, 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15449 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150146+0000", 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150146+0000", 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150146+0000", 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.03e-07 2024-09-14T06:59:44.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150147+0000", 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.167e-06 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150150+0000", 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.242e-06 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150153+0000", 2024-09-14T06:59:44.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191418+0000", 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041265405999999998 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191491+0000", 2024-09-14T06:59:44.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2681000000000004e-05 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191555+0000", 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3588000000000002e-05 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15450 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 7766016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.264 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150156+0000", 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051885223000000001, 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041559637000000003, 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15450 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150156+0000", 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150156+0000", 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150156+0000", 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4299999999999999e-07 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150158+0000", 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1170000000000001e-06 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150161+0000", 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1410000000000001e-06 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150163+0000", 2024-09-14T06:59:44.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.491e-06 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191577+0000", 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041413644999999999 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191650+0000", 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3201999999999995e-05 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191716+0000", 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5797999999999996e-05 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15451 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 7880704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150166+0000", 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051874989000000003, 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041685262000000001, 2024-09-14T06:59:44.269 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15451 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150166+0000", 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150166+0000", 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150167+0000", 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150168+0000", 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1379999999999999e-06 2024-09-14T06:59:44.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150171+0000", 2024-09-14T06:59:44.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0900000000000001e-06 2024-09-14T06:59:44.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150174+0000", 2024-09-14T06:59:44.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8009999999999999e-06 2024-09-14T06:59:44.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191745+0000", 2024-09-14T06:59:44.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041571794000000002 2024-09-14T06:59:44.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191789+0000", 2024-09-14T06:59:44.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769000000000001e-05 2024-09-14T06:59:44.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191852+0000", 2024-09-14T06:59:44.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2459999999999995e-05 2024-09-14T06:59:44.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.275 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.275 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.276 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15452 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 7995392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.276 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150178+0000", 2024-09-14T06:59:44.276 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051863251999999999, 2024-09-14T06:59:44.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041887146, 2024-09-14T06:59:44.276 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.277 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.277 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.277 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.278 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.278 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.278 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15452 2024-09-14T06:59:44.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.278 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150178+0000", 2024-09-14T06:59:44.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150178+0000", 2024-09-14T06:59:44.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150178+0000", 2024-09-14T06:59:44.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:59:44.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150180+0000", 2024-09-14T06:59:44.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.189e-06 2024-09-14T06:59:44.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150183+0000", 2024-09-14T06:59:44.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.168e-06 2024-09-14T06:59:44.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150186+0000", 2024-09-14T06:59:44.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6789999999999998e-06 2024-09-14T06:59:44.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191897+0000", 2024-09-14T06:59:44.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041710710999999998 2024-09-14T06:59:44.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.191986+0000", 2024-09-14T06:59:44.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8447000000000002e-05 2024-09-14T06:59:44.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192065+0000", 2024-09-14T06:59:44.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9727000000000004e-05 2024-09-14T06:59:44.286 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.286 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.286 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.287 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15453 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8110080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.287 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150192+0000", 2024-09-14T06:59:44.288 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051849182000000001, 2024-09-14T06:59:44.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.042095997000000003, 2024-09-14T06:59:44.288 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.288 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.288 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.289 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.289 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.289 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.289 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15453 2024-09-14T06:59:44.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.289 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150192+0000", 2024-09-14T06:59:44.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150192+0000", 2024-09-14T06:59:44.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150371+0000", 2024-09-14T06:59:44.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017915699999999999 2024-09-14T06:59:44.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150376+0000", 2024-09-14T06:59:44.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0990000000000001e-06 2024-09-14T06:59:44.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150383+0000", 2024-09-14T06:59:44.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9689999999999994e-06 2024-09-14T06:59:44.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150390+0000", 2024-09-14T06:59:44.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5570000000000001e-06 2024-09-14T06:59:44.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192111+0000", 2024-09-14T06:59:44.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041720871999999999 2024-09-14T06:59:44.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192216+0000", 2024-09-14T06:59:44.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000105364 2024-09-14T06:59:44.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192288+0000", 2024-09-14T06:59:44.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1978999999999996e-05 2024-09-14T06:59:44.297 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.298 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.298 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15454 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8224768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.298 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150401+0000", 2024-09-14T06:59:44.299 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051640782000000003, 2024-09-14T06:59:44.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.042089948000000002, 2024-09-14T06:59:44.299 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.299 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.299 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.300 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.300 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.300 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.300 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15454 2024-09-14T06:59:44.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.301 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150401+0000", 2024-09-14T06:59:44.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150401+0000", 2024-09-14T06:59:44.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150401+0000", 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150403+0000", 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.154e-06 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150418+0000", 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.452e-05 2024-09-14T06:59:44.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150424+0000", 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8810000000000001e-06 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192333+0000", 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.041909825999999997 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192423+0000", 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9711999999999998e-05 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192491+0000", 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7403000000000005e-05 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15455 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8339456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.305 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.150431+0000", 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.051610096000000001, 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.042288066999999999, 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15455 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.306 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150431+0000", 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150431+0000", 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150487+0000", 2024-09-14T06:59:44.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6097000000000002e-05 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150492+0000", 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9799999999999998e-06 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150503+0000", 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0893999999999999e-05 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.150510+0000", 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2219999999999999e-06 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192550+0000", 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.042040401999999998 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192631+0000", 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0952999999999996e-05 2024-09-14T06:59:44.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.192719+0000", 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8518999999999997e-05 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15659 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 2465792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.159247+0000", 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.042794579999999999, 2024-09-14T06:59:44.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040579090999999998, 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15659 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159247+0000", 2024-09-14T06:59:44.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159247+0000", 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159389+0000", 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014205199999999999 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159393+0000", 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.659e-06 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159401+0000", 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.072e-06 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159407+0000", 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9159999999999998e-06 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.199501+0000", 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040094276999999998 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.199752+0000", 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025122799999999999 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.199826+0000", 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3887000000000006e-05 2024-09-14T06:59:44.314 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15660 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 2580480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.159417+0000", 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.042624687000000001, 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040587435999999998, 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.315 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15660 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159417+0000", 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159417+0000", 2024-09-14T06:59:44.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159417+0000", 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4899999999999995e-07 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159419+0000", 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.199e-06 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159432+0000", 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2927e-05 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159438+0000", 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.541e-06 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.199869+0000", 2024-09-14T06:59:44.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040430964999999999 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.199932+0000", 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3567999999999999e-05 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.200004+0000", 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1687000000000007e-05 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15661 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.159446+0000", 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.042595643000000002, 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040750301000000003, 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.320 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15661 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159446+0000", 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.159446+0000", 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.160894+0000", 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001447877 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.160899+0000", 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9280000000000001e-06 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.160910+0000", 2024-09-14T06:59:44.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.093e-05 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.160918+0000", 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9129999999999992e-06 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.200052+0000", 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.039133953999999999 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.200130+0000", 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7787000000000006e-05 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.200196+0000", 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5912000000000002e-05 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15667 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 3383296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.161028+0000", 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.041013072999999997, 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040386134999999997, 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15667 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.161028+0000", 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.161028+0000", 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.161029+0000", 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.161035+0000", 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.161039+0000", 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.721e-06 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.161042+0000", 2024-09-14T06:59:44.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:59:44.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.201233+0000", 2024-09-14T06:59:44.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040190938000000002 2024-09-14T06:59:44.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.201332+0000", 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8833999999999996e-05 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.201414+0000", 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.297e-05 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.329 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.330 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:44.330 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:59:44.330 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:59:44.330 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-14T06:59:44.330 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:59:44.331 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:59:44.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-14T06:59:44.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:59:44.333 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:44.330+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:44.334 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:59:44.432 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:44.431+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:44.486 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:59:44.486 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:44.528 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:44.529 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:44.529 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:44.529 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:44.533 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:44.532+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:44.540 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:59:44.556 INFO:teuthology.orchestra.run.smithi007.stdout:1019 trunc from 0x5ec4e0f to 0x1b5bb34 2024-09-14T06:59:44.594 INFO:teuthology.orchestra.run.smithi007.stdout:1020 write 0x46b3018 thru 0x46ba4ec (0x74d5 bytes) 2024-09-14T06:59:44.617 INFO:teuthology.orchestra.run.smithi007.stdout:1021 write 0xe0ffec5 thru 0xe10cf58 (0xd094 bytes) 2024-09-14T06:59:44.625 INFO:teuthology.orchestra.run.smithi007.stdout:1023 read 0x9b13e0d thru 0x9b15004 (0x11f8 bytes) 2024-09-14T06:59:44.625 INFO:teuthology.orchestra.run.smithi007.stdout:1024 writesame 0x268c6ac thru 0x268f6a3 data_size 0x14(0x2ff8 bytes) 2024-09-14T06:59:44.628 INFO:teuthology.orchestra.run.smithi007.stdout:1025 read 0x765be64 thru 0x76635d7 (0x7774 bytes) 2024-09-14T06:59:44.628 INFO:teuthology.orchestra.run.smithi007.stdout:1026 writesame 0x2c41efb thru 0x2c48de3 data_size 0x1(0x6ee9 bytes) 2024-09-14T06:59:44.633 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:44.633+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:44.636 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:44.637 INFO:teuthology.orchestra.run.smithi007.stdout:1028 read 0x4c9025 thru 0x4d5fd9 (0xcfb5 bytes) 2024-09-14T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stdout:1029 punch from 0x6ed421a to 0x6ed4dd8, (0xbbe bytes) 2024-09-14T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stdout:1032 read 0x334664b thru 0x3353c92 (0xd648 bytes) 2024-09-14T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stdout:1033 read 0x9a90bc4 thru 0x9a99306 (0x8743 bytes) 2024-09-14T06:59:44.638 INFO:teuthology.orchestra.run.smithi007.stdout:1034 writesame 0xa7886c0 thru 0xa78a6d4 data_size 0x1(0x2015 bytes) 2024-09-14T06:59:44.643 INFO:teuthology.orchestra.run.smithi007.stdout:1035 punch from 0x43a12f7 to 0x43a31c5, (0x1ece bytes) 2024-09-14T06:59:44.643 INFO:teuthology.orchestra.run.smithi007.stdout:1036 writesame 0x38336f5 thru 0x3835270 data_size 0x2(0x1b7c bytes) 2024-09-14T06:59:44.648 INFO:teuthology.orchestra.run.smithi007.stdout:1037 read 0x6d0f951 thru 0x6d1dd23 (0xe3d3 bytes) 2024-09-14T06:59:44.648 INFO:teuthology.orchestra.run.smithi007.stdout:1038 punch from 0x216e40a to 0x217b888, (0xd47e bytes) 2024-09-14T06:59:44.648 INFO:teuthology.orchestra.run.smithi007.stdout:1040 write 0x5620bb6 thru 0x5625a86 (0x4ed1 bytes) 2024-09-14T06:59:44.651 INFO:teuthology.orchestra.run.smithi007.stdout:1041 punch from 0xabf5150 to 0xabfdbd8, (0x8a88 bytes) 2024-09-14T06:59:44.652 INFO:teuthology.orchestra.run.smithi007.stdout:1043 trunc from 0xe10cf59 to 0xdbd1408 2024-09-14T06:59:44.662 INFO:teuthology.orchestra.run.smithi007.stdout:1044 read 0x825db8 thru 0x82d25f (0x74a8 bytes) 2024-09-14T06:59:44.664 INFO:teuthology.orchestra.run.smithi007.stdout:1045 trunc from 0xdbd1408 to 0xc2012a9 2024-09-14T06:59:44.670 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:44.675 INFO:teuthology.orchestra.run.smithi007.stdout:1047 writesame 0xc6944c8 thru 0xc699cc8 data_size 0x1(0x5801 bytes) 2024-09-14T06:59:44.676 INFO:teuthology.orchestra.run.smithi007.stdout:1048 write 0x34b8043 thru 0x34c43a3 (0xc361 bytes) 2024-09-14T06:59:44.690 INFO:teuthology.orchestra.run.smithi007.stdout:1049 read 0xc28cbbc thru 0xc2999f9 (0xce3e bytes) 2024-09-14T06:59:44.690 INFO:teuthology.orchestra.run.smithi007.stdout:1051 write 0xb634e03 thru 0xb63650c (0x170a bytes) 2024-09-14T06:59:44.693 INFO:teuthology.orchestra.run.smithi007.stdout:1052 write 0x40128d2 thru 0x4018ea0 (0x65cf bytes) 2024-09-14T06:59:44.695 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:44.695 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:44.695 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:44.695 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:44.695 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:44.698 INFO:teuthology.orchestra.run.smithi007.stdout:1053 punch from 0x6995163 to 0x69a1d55, (0xcbf2 bytes) 2024-09-14T06:59:44.698 INFO:teuthology.orchestra.run.smithi007.stdout:1054 write 0x9573857 thru 0x9573c94 (0x43e bytes) 2024-09-14T06:59:44.701 INFO:teuthology.orchestra.run.smithi007.stdout:1055 write 0x2ff945a thru 0x2fff0e4 (0x5c8b bytes) 2024-09-14T06:59:44.703 INFO:teuthology.orchestra.run.smithi007.stdout:1056 punch from 0x133e1c1 to 0x134c61c, (0xe45b bytes) 2024-09-14T06:59:44.703 INFO:teuthology.orchestra.run.smithi007.stdout:1057 write 0xa58bbd6 thru 0xa596085 (0xa4b0 bytes) 2024-09-14T06:59:44.705 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:59:44.713 INFO:teuthology.orchestra.run.smithi007.stdout:1058 writesame 0x51deb82 thru 0x51e4e2c data_size 0x1d(0x62ab bytes) 2024-09-14T06:59:44.722 INFO:teuthology.orchestra.run.smithi007.stdout:1059 trunc from 0xc699cc9 to 0x942b531 2024-09-14T06:59:44.738 INFO:teuthology.orchestra.run.smithi007.stdout:1060 write 0x23f2a93 thru 0x23fb1de (0x874c bytes) 2024-09-14T06:59:44.742 INFO:teuthology.orchestra.run.smithi007.stdout:1062 punch from 0x42a8b8 to 0x430bdc, (0x6324 bytes) 2024-09-14T06:59:44.743 INFO:teuthology.orchestra.run.smithi007.stdout:1063 read 0x1dfcfa9 thru 0x1e09919 (0xc971 bytes) 2024-09-14T06:59:44.747 INFO:teuthology.orchestra.run.smithi007.stdout:1064 trunc from 0x942b531 to 0x53f1330 2024-09-14T06:59:44.748 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:44.748+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:44.776 INFO:teuthology.orchestra.run.smithi007.stdout:1066 writesame 0x91946d9 thru 0x91a2944 data_size 0x4(0xe26c bytes) 2024-09-14T06:59:44.786 INFO:teuthology.orchestra.run.smithi007.stdout:1067 write 0xc61d076 thru 0xc620cd7 (0x3c62 bytes) 2024-09-14T06:59:44.790 INFO:teuthology.orchestra.run.smithi007.stdout:1070 clone 7 order 18 su 8192 sc 9 2024-09-14T06:59:44.817 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T06:59:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.051260871, 2024-09-14T06:59:44.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:44.822 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.9781416050000002, 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T06:59:44.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.828 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.9778849520000001, 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.829 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T06:59:44.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.9515978780000003, 2024-09-14T06:59:44.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T06:59:44.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T06:59:44.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T06:59:44.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.838 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.9514732639999997, 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:44.839 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:44.840 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T06:59:44.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.840 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T06:59:44.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4568541020000003, 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.844 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T06:59:44.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.1824514419999996, 2024-09-14T06:59:44.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T06:59:44.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.854 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.187835832, 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.859 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.99766336600000005, 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T06:59:44.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.94853007600000006, 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.864 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T06:59:44.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T06:59:44.868 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.58031111199999996, 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.869 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T06:59:44.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T06:59:44.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.874 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.874 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T06:59:44.874 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.45974217299999998, 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T06:59:44.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T06:59:44.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.32882629200000002, 2024-09-14T06:59:44.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T06:59:44.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T06:59:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.29978903000000001, 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:59:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T06:59:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.27401960800000003, 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T06:59:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T06:59:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T06:59:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T06:59:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T06:59:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T06:59:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T06:59:44.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T06:59:44.894 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.24066146499999999, 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T06:59:44.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T06:59:44.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T06:59:44.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T06:59:44.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.899 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.076453683999999994, 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.065074424000000006, 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T06:59:44.904 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T06:59:44.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T06:59:44.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T06:59:44.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T06:59:44.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T06:59:44.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.065026730000000005, 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.909 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T06:59:44.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.914 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.064684405, 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:44.915 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T06:59:44.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T06:59:44.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.920 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:44.920 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:44.920 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:44.921 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:59:44.922 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:44.920+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:45.022 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:45.021+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:45.066 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:45.066 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:45.066 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:45.066 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:45.075 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:59:45.123 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:45.122+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:45.177 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:45.177 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:45.177 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:45.177 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:45.177 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:45.185 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:59:45.229 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:45.228+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:45.290 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:45.290 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:45.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:45.290 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:45.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.290 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8489 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8208384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.290 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670275+0000", 2024-09-14T06:59:45.290 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.617873863, 2024-09-14T06:59:45.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033127139, 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8489 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670276+0000", 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670277+0000", 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6109999999999999e-06 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670283+0000", 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7409999999999998e-06 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670287+0000", 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1679999999999997e-06 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703311+0000", 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033023640999999999 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703352+0000", 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1168999999999998e-05 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703402+0000", 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0424999999999999e-05 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8490 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.61785628, 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033303263, 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.295 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8490 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7800000000000002e-07 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670295+0000", 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670300+0000", 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5369999999999997e-06 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670304+0000", 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8789999999999996e-06 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703433+0000", 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033128697999999998 2024-09-14T06:59:45.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703510+0000", 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6662000000000006e-05 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703596+0000", 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6614999999999997e-05 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.299 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8491 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8454144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6178386499999999, 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033389611999999999, 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8491 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670317+0000", 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.19e-06 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670323+0000", 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670331+0000", 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1280000000000008e-06 2024-09-14T06:59:45.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703611+0000", 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033279557000000001 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703650+0000", 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9410999999999999e-05 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703700+0000", 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0117000000000001e-05 2024-09-14T06:59:45.303 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8492 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8568832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6178105999999999, 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033561793999999999, 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.304 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8492 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670341+0000", 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8700000000000001e-06 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670347+0000", 2024-09-14T06:59:45.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670351+0000", 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4660000000000001e-06 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703731+0000", 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033379502999999998 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703812+0000", 2024-09-14T06:59:45.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1192000000000005e-05 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703900+0000", 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8382999999999997e-05 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8493 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8683520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6177921020000001, 2024-09-14T06:59:45.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033663516999999997, 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8493 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670359+0000", 2024-09-14T06:59:45.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.685e-06 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670365+0000", 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.57e-06 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670369+0000", 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4460000000000003e-06 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703928+0000", 2024-09-14T06:59:45.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033558530000000003 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703970+0000", 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1909999999999997e-05 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704021+0000", 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0988e-05 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8494 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8798208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.617762506, 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033828579999999997, 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8494 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670389+0000", 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.64e-06 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670393+0000", 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670396+0000", 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6369999999999999e-06 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704051+0000", 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033655293000000003 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704128+0000", 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7453000000000005e-05 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704215+0000", 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6884000000000004e-05 2024-09-14T06:59:45.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8495 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8912896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670400+0000", 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.617748956, 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033930725000000002, 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.317 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8495 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670401+0000", 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.15e-07 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670402+0000", 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.850000000000001e-07 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670405+0000", 2024-09-14T06:59:45.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.225e-06 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670407+0000", 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.34e-06 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704240+0000", 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033833019999999998 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704281+0000", 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1332999999999999e-05 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704331+0000", 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9506999999999999e-05 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8496 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9027584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.321 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6177376319999999, 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034114617, 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8496 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:45.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670413+0000", 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8999999999999995e-07 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670416+0000", 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2729999999999998e-06 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670418+0000", 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3709999999999998e-06 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704362+0000", 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033943917999999997 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704439+0000", 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6922999999999995e-05 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704526+0000", 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7035999999999998e-05 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.325 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8497 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9142272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670421+0000", 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.617727744, 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034220315000000001, 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.326 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8497 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670422+0000", 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670426+0000", 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5029999999999999e-06 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670430+0000", 2024-09-14T06:59:45.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670432+0000", 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4269999999999998e-06 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704552+0000", 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034120277999999997 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704592+0000", 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9884e-05 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704642+0000", 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9623999999999998e-05 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8498 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9256960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.330 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6177139810000001, 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034407434000000001, 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8498 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000001e-07 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670436+0000", 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999996e-07 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670439+0000", 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670442+0000", 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3120000000000001e-06 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704673+0000", 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034231175000000003 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704749+0000", 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6069999999999995e-05 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704843+0000", 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3622000000000006e-05 2024-09-14T06:59:45.334 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8499 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9371648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.617704461, 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034523731000000002, 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.335 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8499 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670446+0000", 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670449+0000", 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809999999999998e-06 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670455+0000", 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7930000000000003e-06 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704869+0000", 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034413931000000002 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704910+0000", 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1032999999999998e-05 2024-09-14T06:59:45.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704968+0000", 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8897999999999998e-05 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8500 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9486336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6176911839999999, 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034708731, 2024-09-14T06:59:45.339 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8500 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670459+0000", 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2999999999999999e-07 2024-09-14T06:59:45.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670462+0000", 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1350000000000001e-06 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670470+0000", 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7400000000000004e-06 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705002+0000", 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034531470000000002 2024-09-14T06:59:45.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705079+0000", 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7472999999999995e-05 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705167+0000", 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7750000000000005e-05 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8501 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9601024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670476+0000", 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6176727550000001, 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034806278000000003, 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8501 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670477+0000", 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.44e-07 2024-09-14T06:59:45.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670479+0000", 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7740000000000001e-06 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670485+0000", 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9150000000000001e-06 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670489+0000", 2024-09-14T06:59:45.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5469999999999998e-06 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705191+0000", 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034701896000000003 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705234+0000", 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2394000000000001e-05 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705283+0000", 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9208e-05 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8502 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9715712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6176524159999999, 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034979270999999999, 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.348 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8502 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:45.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670506+0000", 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9740000000000008e-06 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670512+0000", 2024-09-14T06:59:45.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670517+0000", 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7049999999999998e-06 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705313+0000", 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034795858999999998 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705391+0000", 2024-09-14T06:59:45.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8166999999999999e-05 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705476+0000", 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5285999999999996e-05 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8503 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9830400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670523+0000", 2024-09-14T06:59:45.352 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.617625994, 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053094, 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8503 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670524+0000", 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670525+0000", 2024-09-14T06:59:45.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.581e-06 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670531+0000", 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8969999999999996e-06 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670536+0000", 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4630000000000003e-06 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705488+0000", 2024-09-14T06:59:45.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034952451000000002 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705526+0000", 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8396999999999998e-05 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705576+0000", 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9926000000000003e-05 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8504 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9945088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6176076560000001, 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035228601999999998, 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8504 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.357 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670543+0000", 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5820000000000001e-06 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670549+0000", 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670554+0000", 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3699999999999997e-06 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705607+0000", 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053594 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705685+0000", 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8102999999999994e-05 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705770+0000", 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4678000000000005e-05 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8505 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10059776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670559+0000", 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6175898690000001, 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035318548999999998, 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.361 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8505 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670560+0000", 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3599999999999999e-07 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670561+0000", 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670568+0000", 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9789999999999996e-06 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670572+0000", 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2509999999999996e-06 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705784+0000", 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035211724 2024-09-14T06:59:45.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705827+0000", 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3276999999999999e-05 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705878+0000", 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0488000000000002e-05 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.365 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8506 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10174464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670579+0000", 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.617569783, 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035495512999999999, 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8506 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670580+0000", 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670582+0000", 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670587+0000", 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.767e-06 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670592+0000", 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2819999999999998e-06 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705909+0000", 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.03531753 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705987+0000", 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706075+0000", 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7998999999999996e-05 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.369 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8507 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10289152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670597+0000", 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6175519569999999, 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035582134000000001, 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.370 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8507 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670598+0000", 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9900000000000002e-07 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670599+0000", 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6029999999999999e-06 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670609+0000", 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.004e-05 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670614+0000", 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0749999999999997e-06 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706089+0000", 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035474358999999997 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706129+0000", 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0204999999999999e-05 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706179+0000", 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0553000000000002e-05 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8508 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10403840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.6175283280000001, 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035743559000000001, 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:45.374 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8508 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:45.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2800000000000002e-07 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670623+0000", 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670629+0000", 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.02e-06 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670634+0000", 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6280000000000004e-06 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706200+0000", 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035566564000000002 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706276+0000", 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5801000000000002e-05 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706364+0000", 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8176000000000004e-05 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:45.378 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T06:59:45.380 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:45.379+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:45.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:59:45.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:59:45.480 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:45.480+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:45.510 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-14T06:59:45.519 INFO:teuthology.orchestra.run.smithi007.stdout:1071 writesame 0xba885e7 thru 0xba900b8 data_size 0x4f(0x7ad2 bytes) 2024-09-14T06:59:45.531 INFO:teuthology.orchestra.run.smithi007.stdout:1073 write 0x9b22836 thru 0x9b26335 (0x3b00 bytes) 2024-09-14T06:59:45.543 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:45.543 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:45.543 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:45.543 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:45.548 INFO:teuthology.orchestra.run.smithi007.stdout:1074 trunc from 0xc620cd8 to 0xebc3f2 2024-09-14T06:59:45.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:59:45.576 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:45.576 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:45.581 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:45.580+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:45.582 INFO:teuthology.orchestra.run.smithi007.stdout:1075 trunc from 0xebc3f2 to 0x1676839 2024-09-14T06:59:45.594 INFO:teuthology.orchestra.run.smithi007.stdout:1076 write 0x1cb22d0 thru 0x1cb5b42 (0x3873 bytes) 2024-09-14T06:59:45.597 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:45.603 INFO:teuthology.orchestra.run.smithi007.stdout:1077 trunc from 0x1cb5b43 to 0x6971d6d 2024-09-14T06:59:45.616 INFO:teuthology.orchestra.run.smithi007.stdout:1079 write 0x83d7cf1 thru 0x83df929 (0x7c39 bytes) 2024-09-14T06:59:45.617 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:45.617 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:45.623 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:45.623 INFO:teuthology.orchestra.run.smithi007.stdout:1082 clone 8 order 18 su 1024 sc 3 2024-09-14T06:59:45.637 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:45.637 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:45.654 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:45.655 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:45.672 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:45.672 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:45.682 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:45.682+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:45.684 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:45.684 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:45.689 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:45.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:59:45.706 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:45.706 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:45.783 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:45.782+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:45.813 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:45.813 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:45.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:59:45.906 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:45.905+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:45.969 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:45.969 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:45.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:59:45.985 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:45.984+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:46.085 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:46.085+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:46.125 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:46.125 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:46.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:59:46.217 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:46.216+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:46.280 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:46.280 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:46.287 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:46.287+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:46.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:59:46.372 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone7 from 0x83df92a (overlap 0xebc3f2) to 0xbc9cc1 2024-09-14T06:59:46.388 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:46.387+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:46.455 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-14T06:59:46.456 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:46.456 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:46.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:59:46.510 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:46.509+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:46.591 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:46.590+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:46.623 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:46.623 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:46.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:59:46.685 INFO:teuthology.orchestra.run.smithi007.stdout:1084 read 0xd58729 thru 0xd5ac6c (0x2544 bytes) 2024-09-14T06:59:46.687 INFO:teuthology.orchestra.run.smithi007.stdout:1085 read 0x6da664e thru 0x6daeffd (0x89b0 bytes) 2024-09-14T06:59:46.689 INFO:teuthology.orchestra.run.smithi007.stdout:1088 trunc from 0x83df92a to 0x9347152 2024-09-14T06:59:46.692 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:46.691+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:46.698 INFO:teuthology.orchestra.run.smithi007.stdout:1089 write 0x256e9b0 thru 0x257e985 (0xffd6 bytes) 2024-09-14T06:59:46.705 INFO:teuthology.orchestra.run.smithi007.stdout:1090 read 0x729fd84 thru 0x72ae72a (0xe9a7 bytes) 2024-09-14T06:59:46.711 INFO:teuthology.orchestra.run.smithi007.stdout:1091 write 0xa392752 thru 0xa392a7a (0x329 bytes) 2024-09-14T06:59:46.724 INFO:teuthology.orchestra.run.smithi007.stdout:1092 trunc from 0xa392a7b to 0x24ca017 2024-09-14T06:59:46.751 INFO:teuthology.orchestra.run.smithi007.stdout:1093 write 0xac18926 thru 0xac22924 (0x9fff bytes) 2024-09-14T06:59:46.755 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:46.755 INFO:teuthology.orchestra.run.smithi007.stdout:1097 read 0x2c96b2 thru 0x2d9436 (0xfd85 bytes) 2024-09-14T06:59:46.758 INFO:teuthology.orchestra.run.smithi007.stdout:1101 read 0x1ee5357 thru 0x1ef13d3 (0xc07d bytes) 2024-09-14T06:59:46.758 INFO:teuthology.orchestra.run.smithi007.stdout:1102 read 0x5adecb4 thru 0x5ae8fc3 (0xa310 bytes) 2024-09-14T06:59:46.766 INFO:teuthology.orchestra.run.smithi007.stdout:1106 write 0xd70a7a8 thru 0xd71632f (0xbb88 bytes) 2024-09-14T06:59:46.771 INFO:teuthology.orchestra.run.smithi007.stdout:1108 read 0x36d9491 thru 0x36d9760 (0x2d0 bytes) 2024-09-14T06:59:46.771 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:46.771 INFO:teuthology.orchestra.run.smithi007.stdout:1110 punch from 0x9b3564 to 0x9b796e, (0x440a bytes) 2024-09-14T06:59:46.771 INFO:teuthology.orchestra.run.smithi007.stdout:1111 punch from 0x7cc93b6 to 0x7cd61ac, (0xcdf6 bytes) 2024-09-14T06:59:46.772 INFO:teuthology.orchestra.run.smithi007.stdout:1112 writesame 0xac75dde thru 0xac7b5b4 data_size 0x1(0x57d7 bytes) 2024-09-14T06:59:46.790 INFO:teuthology.orchestra.run.smithi007.stdout:1113 trunc from 0xd716330 to 0x1f1cea8 2024-09-14T06:59:46.793 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:46.792+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:46.797 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:46.797 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:46.805 INFO:teuthology.orchestra.run.smithi007.stdout:1114 clone 9 order 21 su 524288 sc 12 2024-09-14T06:59:46.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:59:46.894 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:46.893+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:46.955 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:46.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:59:46.995 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:46.995+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:47.069 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:47.070 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:47.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:59:47.121 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:47.121+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:47.185 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:47.185 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:47.194 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:59:47.241 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:47.238+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:47.298 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:47.298+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:47.300 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:47.300 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:47.309 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:59:47.374 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone8 from 0x1f1cea8 (overlap 0x1f1cea8) to 0x0 2024-09-14T06:59:47.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:59:47.432 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:47.429+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:47.474 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:47.474 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:47.484 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:59:47.529 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:47.528+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:47.601 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:47.601+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:47.618 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:47.618 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:47.629 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:59:47.708 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:47.703+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:47.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:59:47.804 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:47.804 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:47.810 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:47.803+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:47.814 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:59:47.905 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:47.904+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:47.944 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:47.945 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:47.954 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:59:47.986 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:47.987 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:48.007 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:48.006+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:48.022 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:48.022 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:48.053 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:48.054 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:48.083 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:48.107 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:48.106+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:48.114 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:48.117 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:48.117 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:48.127 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:59:48.137 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:48.137 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:48.155 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:48.155 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:48.175 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:48.175 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:48.189 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-14T06:59:48.208 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:48.207+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:48.249 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:48.249 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:48.250 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:48.250 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:48.258 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:59:48.273 INFO:teuthology.orchestra.run.smithi007.stdout:1115 write 0x3831b87 thru 0x3840d74 (0xf1ee bytes) 2024-09-14T06:59:48.281 INFO:teuthology.orchestra.run.smithi007.stdout:1116 writesame 0x3aa0b00 thru 0x3aad679 data_size 0x5(0xcb7a bytes) 2024-09-14T06:59:48.294 INFO:teuthology.orchestra.run.smithi007.stdout:1117 write 0x97f264d thru 0x97f43ee (0x1da2 bytes) 2024-09-14T06:59:48.309 INFO:teuthology.orchestra.run.smithi007.stdout:1118 writesame 0xe4eaa73 thru 0xe4eeab9 data_size 0x5(0x4047 bytes) 2024-09-14T06:59:48.309 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:48.309 INFO:teuthology.orchestra.run.smithi007.stdout:1120 write 0x480791e thru 0x4815039 (0xd71c bytes) 2024-09-14T06:59:48.309 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:48.308+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:48.312 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:48.313 INFO:teuthology.orchestra.run.smithi007.stdout:1122 trunc from 0xe4eeaba to 0xd204328 2024-09-14T06:59:48.316 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T06:59:48.327 INFO:teuthology.orchestra.run.smithi007.stdout:1123 write 0x31b41c5 thru 0x31b9407 (0x5243 bytes) 2024-09-14T06:59:48.332 INFO:teuthology.orchestra.run.smithi007.stdout:1124 punch from 0xbcf3a28 to 0xbcf9118, (0x56f0 bytes) 2024-09-14T06:59:48.335 INFO:teuthology.orchestra.run.smithi007.stdout:1125 trunc from 0xd204328 to 0xa7ce04a 2024-09-14T06:59:48.341 INFO:teuthology.orchestra.run.smithi007.stdout:1127 punch from 0x960b142 to 0x961a6ca, (0xf588 bytes) 2024-09-14T06:59:48.341 INFO:teuthology.orchestra.run.smithi007.stdout:1128 write 0x5e4d89d thru 0x5e4efa3 (0x1707 bytes) 2024-09-14T06:59:48.345 INFO:teuthology.orchestra.run.smithi007.stdout:1130 writesame 0x21f8645 thru 0x22030af data_size 0x1(0xaa6b bytes) 2024-09-14T06:59:48.391 INFO:teuthology.orchestra.run.smithi007.stdout:1131 trunc from 0xa7ce04a to 0x6c90da4 2024-09-14T06:59:48.403 INFO:teuthology.orchestra.run.smithi007.stdout:1132 punch from 0x2881dbe to 0x2882a2d, (0xc6f bytes) 2024-09-14T06:59:48.403 INFO:teuthology.orchestra.run.smithi007.stdout:1133 read 0x60d1d24 thru 0x60dd5ba (0xb897 bytes) 2024-09-14T06:59:48.403 INFO:teuthology.orchestra.run.smithi007.stdout:1134 writesame 0x1e05906 thru 0x1e0b680 data_size 0x3(0x5d7b bytes) 2024-09-14T06:59:48.410 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:48.410+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:48.412 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:48.412 INFO:teuthology.orchestra.run.smithi007.stdout:1136 read 0x413d1e thru 0x41b1ea (0x74cd bytes) 2024-09-14T06:59:48.418 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:48.418 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:48.418 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:48.418 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:48.418 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:48.426 INFO:teuthology.orchestra.run.smithi007.stdout:1137 writesame 0xd5e47bb thru 0xd5e8c80 data_size 0x1(0x44c6 bytes) 2024-09-14T06:59:48.426 INFO:teuthology.orchestra.run.smithi007.stdout:1138 trunc from 0xd5e8c81 to 0xd362f38 2024-09-14T06:59:48.427 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:59:48.431 INFO:teuthology.orchestra.run.smithi007.stdout:1139 write 0x9e0a150 thru 0x9e13626 (0x94d7 bytes) 2024-09-14T06:59:48.435 INFO:teuthology.orchestra.run.smithi007.stdout:1141 punch from 0xa14e59c to 0xa15bd0f, (0xd773 bytes) 2024-09-14T06:59:48.443 INFO:teuthology.orchestra.run.smithi007.stdout:1143 trunc from 0xd362f38 to 0x67cb4cf 2024-09-14T06:59:48.453 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:48.461 INFO:teuthology.orchestra.run.smithi007.stdout:1145 trunc from 0x67cb4cf to 0xdfca07d 2024-09-14T06:59:48.476 INFO:teuthology.orchestra.run.smithi007.stdout:1146 trunc from 0xdfca07d to 0x5d87d6f 2024-09-14T06:59:48.496 INFO:teuthology.orchestra.run.smithi007.stdout:1147 write 0xc92a72f thru 0xc92cadd (0x23af bytes) 2024-09-14T06:59:48.499 INFO:teuthology.orchestra.run.smithi007.stdout:1148 writesame 0x7e0c02b thru 0x7e1ab83 data_size 0x85(0xeb59 bytes) 2024-09-14T06:59:48.503 INFO:teuthology.orchestra.run.smithi007.stdout:1151 trunc from 0xc92cade to 0xe12e095 2024-09-14T06:59:48.509 INFO:teuthology.orchestra.run.smithi007.stdout:1153 writesame 0x443ae26 thru 0x443ec85 data_size 0x1(0x3e60 bytes) 2024-09-14T06:59:48.511 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:48.511+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:48.523 INFO:teuthology.orchestra.run.smithi007.stdout:1154 write 0xed3813d thru 0xed445e6 (0xc4aa bytes) 2024-09-14T06:59:48.525 INFO:teuthology.orchestra.run.smithi007.stdout:1155 clone 10 order 23 su 4194304 sc 12 2024-09-14T06:59:48.538 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.773297631999998, 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.539 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T06:59:48.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:48.543 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.7001783659999994, 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:48.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:48.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T06:59:48.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T06:59:48.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:48.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T06:59:48.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T06:59:48.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.549 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6999217130000002, 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6736346389999994, 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.554 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T06:59:48.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:59:48.558 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6735100250000006, 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T06:59:48.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.1788908629999995, 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T06:59:48.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T06:59:48.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 7.9044882029999997, 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T06:59:48.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T06:59:48.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T06:59:48.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T06:59:48.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T06:59:48.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T06:59:48.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.9098725930000002, 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T06:59:48.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.7197001270000003, 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T06:59:48.576 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.580 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.670566837, 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.584 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.3023478730000004, 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.1817789340000004, 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.589 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T06:59:48.593 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.0508630529999996, 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T06:59:48.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.597 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 4.0218257910000004, 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T06:59:48.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T06:59:48.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.9960563690000002, 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.602 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T06:59:48.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.606 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.9626982260000001, 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T06:59:48.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T06:59:48.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.7984904450000001, 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T06:59:48.611 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T06:59:48.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T06:59:48.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T06:59:48.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.7871111850000001, 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T06:59:48.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T06:59:48.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.7870634910000001, 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T06:59:48.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T06:59:48.624 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.786721166, 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:48.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T06:59:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T06:59:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:48.629 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:48.629 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:48.630 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:59:48.631 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:48.630+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:48.732 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:48.731+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:48.776 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:48.776 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:48.777 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:48.777 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:48.785 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:59:48.869 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:48.868+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:48.929 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:48.929 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:48.929 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:48.929 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:48.929 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:48.933 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:48.933+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:48.938 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:59:49.035 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:49.034+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:49.083 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8489 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8208384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670275+0000", 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4120103740000003, 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033127139, 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.084 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8489 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670276+0000", 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670277+0000", 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6109999999999999e-06 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670283+0000", 2024-09-14T06:59:49.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7409999999999998e-06 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670287+0000", 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1679999999999997e-06 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703311+0000", 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033023640999999999 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703352+0000", 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1168999999999998e-05 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703402+0000", 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0424999999999999e-05 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8490 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.088 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4119927910000003, 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033303263, 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8490 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7800000000000002e-07 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670295+0000", 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670300+0000", 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5369999999999997e-06 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670304+0000", 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8789999999999996e-06 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703433+0000", 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033128697999999998 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703510+0000", 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6662000000000006e-05 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703596+0000", 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6614999999999997e-05 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.092 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8491 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8454144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.411975161, 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033389611999999999, 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.093 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8491 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670317+0000", 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.19e-06 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670323+0000", 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670331+0000", 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1280000000000008e-06 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703611+0000", 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033279557000000001 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703650+0000", 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9410999999999999e-05 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703700+0000", 2024-09-14T06:59:49.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0117000000000001e-05 2024-09-14T06:59:49.097 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.097 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.097 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.097 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8492 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8568832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.097 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4119471109999999, 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033561793999999999, 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8492 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:59:49.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670341+0000", 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8700000000000001e-06 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670347+0000", 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.100 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670351+0000", 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4660000000000001e-06 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703731+0000", 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033379502999999998 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703812+0000", 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1192000000000005e-05 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703900+0000", 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8382999999999997e-05 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8493 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8683520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4119286129999997, 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033663516999999997, 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.102 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8493 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:49.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670359+0000", 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.685e-06 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670365+0000", 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.57e-06 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670369+0000", 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4460000000000003e-06 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703928+0000", 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033558530000000003 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703970+0000", 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1909999999999997e-05 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704021+0000", 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0988e-05 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.106 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8494 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8798208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4118990169999996, 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033828579999999997, 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.107 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8494 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670389+0000", 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.64e-06 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670393+0000", 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:59:49.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670396+0000", 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6369999999999999e-06 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704051+0000", 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033655293000000003 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704128+0000", 2024-09-14T06:59:49.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7453000000000005e-05 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704215+0000", 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6884000000000004e-05 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8495 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8912896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670400+0000", 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4118854670000003, 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033930725000000002, 2024-09-14T06:59:49.111 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8495 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670401+0000", 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.15e-07 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670402+0000", 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.850000000000001e-07 2024-09-14T06:59:49.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670405+0000", 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.225e-06 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670407+0000", 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.34e-06 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704240+0000", 2024-09-14T06:59:49.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033833019999999998 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704281+0000", 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1332999999999999e-05 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704331+0000", 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9506999999999999e-05 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8496 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9027584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4118741430000004, 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034114617, 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8496 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.116 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:59:49.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670413+0000", 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8999999999999995e-07 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670416+0000", 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2729999999999998e-06 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670418+0000", 2024-09-14T06:59:49.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3709999999999998e-06 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704362+0000", 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033943917999999997 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704439+0000", 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6922999999999995e-05 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704526+0000", 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7035999999999998e-05 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8497 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9142272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670421+0000", 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4118642550000002, 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034220315000000001, 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.120 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8497 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670422+0000", 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670426+0000", 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5029999999999999e-06 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670430+0000", 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670432+0000", 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4269999999999998e-06 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704552+0000", 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034120277999999997 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704592+0000", 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9884e-05 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704642+0000", 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9623999999999998e-05 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.124 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8498 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9256960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4118504920000001, 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034407434000000001, 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8498 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000001e-07 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670436+0000", 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999996e-07 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670439+0000", 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670442+0000", 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3120000000000001e-06 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704673+0000", 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034231175000000003 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704749+0000", 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6069999999999995e-05 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704843+0000", 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3622000000000006e-05 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.128 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8499 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9371648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4118409720000002, 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034523731000000002, 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.129 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8499 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670446+0000", 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670449+0000", 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809999999999998e-06 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670455+0000", 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7930000000000003e-06 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704869+0000", 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034413931000000002 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704910+0000", 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1032999999999998e-05 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704968+0000", 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8897999999999998e-05 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8500 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9486336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4118276950000004, 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034708731, 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.133 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8500 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670459+0000", 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2999999999999999e-07 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670462+0000", 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1350000000000001e-06 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670470+0000", 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7400000000000004e-06 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705002+0000", 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034531470000000002 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705079+0000", 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7472999999999995e-05 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705167+0000", 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7750000000000005e-05 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8501 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9601024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670476+0000", 2024-09-14T06:59:49.137 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4118092659999997, 2024-09-14T06:59:49.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034806278000000003, 2024-09-14T06:59:49.138 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.138 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.138 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.138 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.138 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.138 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.138 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8501 2024-09-14T06:59:49.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670477+0000", 2024-09-14T06:59:49.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.44e-07 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670479+0000", 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7740000000000001e-06 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670485+0000", 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9150000000000001e-06 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670489+0000", 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5469999999999998e-06 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705191+0000", 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034701896000000003 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705234+0000", 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2394000000000001e-05 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705283+0000", 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9208e-05 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8502 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9715712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4117889269999999, 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034979270999999999, 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.142 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8502 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670506+0000", 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9740000000000008e-06 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670512+0000", 2024-09-14T06:59:49.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670517+0000", 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7049999999999998e-06 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705313+0000", 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034795858999999998 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705391+0000", 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8166999999999999e-05 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705476+0000", 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5285999999999996e-05 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8503 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9830400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670523+0000", 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4117625050000004, 2024-09-14T06:59:49.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053094, 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8503 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670524+0000", 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670525+0000", 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.581e-06 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670531+0000", 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8969999999999996e-06 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670536+0000", 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4630000000000003e-06 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705488+0000", 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034952451000000002 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705526+0000", 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8396999999999998e-05 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705576+0000", 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9926000000000003e-05 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8504 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9945088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.150 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4117441670000002, 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035228601999999998, 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8504 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:49.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670543+0000", 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5820000000000001e-06 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670549+0000", 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670554+0000", 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3699999999999997e-06 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705607+0000", 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053594 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705685+0000", 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8102999999999994e-05 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705770+0000", 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4678000000000005e-05 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8505 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10059776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670559+0000", 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4117263800000002, 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035318548999999998, 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8505 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.155 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670560+0000", 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3599999999999999e-07 2024-09-14T06:59:49.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670561+0000", 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670568+0000", 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9789999999999996e-06 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670572+0000", 2024-09-14T06:59:49.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2509999999999996e-06 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705784+0000", 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035211724 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705827+0000", 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3276999999999999e-05 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705878+0000", 2024-09-14T06:59:49.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0488000000000002e-05 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8506 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10174464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670579+0000", 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.411706294, 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035495512999999999, 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.159 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8506 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670580+0000", 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670582+0000", 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670587+0000", 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.767e-06 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670592+0000", 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2819999999999998e-06 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705909+0000", 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.03531753 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705987+0000", 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706075+0000", 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7998999999999996e-05 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8507 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10289152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670597+0000", 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4116884680000004, 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035582134000000001, 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.163 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8507 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670598+0000", 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9900000000000002e-07 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670599+0000", 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6029999999999999e-06 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670609+0000", 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.004e-05 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670614+0000", 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0749999999999997e-06 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706089+0000", 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035474358999999997 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706129+0000", 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0204999999999999e-05 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706179+0000", 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0553000000000002e-05 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8508 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10403840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 5.4116648390000002, 2024-09-14T06:59:49.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035743559000000001, 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8508 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2800000000000002e-07 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670623+0000", 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:59:49.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670629+0000", 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.02e-06 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670634+0000", 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6280000000000004e-06 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706200+0000", 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035566564000000002 2024-09-14T06:59:49.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706276+0000", 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5801000000000002e-05 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706364+0000", 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8176000000000004e-05 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:49.171 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:49.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:59:49.173 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:49.172+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:49.274 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:49.273+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:49.293 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:49.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:59:49.375 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:49.375+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:49.390 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone9 from 0xed445e7 (overlap 0x1f1cea8) to 0x955798 2024-09-14T06:59:49.437 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:49.437 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:49.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:59:49.476 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:49.475+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:49.499 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:59:49.499 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:59:49.500 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:59:49.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-14T06:59:49.577 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:49.576+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:49.633 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40601 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 778240~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.633 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210449+0000", 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419963477, 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074680440000000001, 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40601 2024-09-14T06:59:49.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210449+0000", 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210449+0000", 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210449+0000", 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210451+0000", 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.81e-06 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210457+0000", 2024-09-14T06:59:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.857e-06 2024-09-14T06:59:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210462+0000", 2024-09-14T06:59:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7330000000000003e-06 2024-09-14T06:59:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.217754+0000", 2024-09-14T06:59:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072918159999999996 2024-09-14T06:59:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.217818+0000", 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4282999999999994e-05 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.217917+0000", 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9168999999999999e-05 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40632 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 729088~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210508+0000", 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419905003, 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075811389999999998, 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40632 2024-09-14T06:59:49.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210508+0000", 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210508+0000", 2024-09-14T06:59:49.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210508+0000", 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5499999999999999e-07 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210510+0000", 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.821e-06 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210517+0000", 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3869999999999997e-06 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210522+0000", 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2510000000000002e-06 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.217945+0000", 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074229550000000002 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218011+0000", 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5308999999999994e-05 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218089+0000", 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8060999999999997e-05 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40633 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 802816~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210529+0000", 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419883644, 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007738511, 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40633 2024-09-14T06:59:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210529+0000", 2024-09-14T06:59:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210529+0000", 2024-09-14T06:59:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210529+0000", 2024-09-14T06:59:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7099999999999997e-07 2024-09-14T06:59:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210531+0000", 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9410000000000001e-06 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210539+0000", 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8329999999999994e-06 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210544+0000", 2024-09-14T06:59:49.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218116+0000", 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007572334 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218182+0000", 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5399000000000002e-05 2024-09-14T06:59:49.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218267+0000", 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40664 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 696320~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.651 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210593+0000", 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4198197889999999, 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078560379999999992, 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40664 2024-09-14T06:59:49.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210593+0000", 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210593+0000", 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210593+0000", 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210595+0000", 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210601+0000", 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0329999999999997e-06 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210606+0000", 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9969999999999998e-06 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218297+0000", 2024-09-14T06:59:49.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0076905790000000003 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218366+0000", 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9516000000000003e-05 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218449+0000", 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2466999999999995e-05 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40665 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 770048~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210612+0000", 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419800226, 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080227230000000007, 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40665 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210612+0000", 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210612+0000", 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210613+0000", 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210618+0000", 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4750000000000001e-06 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210625+0000", 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3350000000000003e-06 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210630+0000", 2024-09-14T06:59:49.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218477+0000", 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078475030000000005 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218544+0000", 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6506000000000003e-05 2024-09-14T06:59:49.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218635+0000", 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1481e-05 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40674 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210638+0000", 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4197749449999999, 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081783970000000004, 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40674 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.663 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210638+0000", 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210638+0000", 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210638+0000", 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-07 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210641+0000", 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.399e-06 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210648+0000", 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4810000000000003e-06 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210653+0000", 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218666+0000", 2024-09-14T06:59:49.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080128960000000003 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218734+0000", 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8541999999999995e-05 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218816+0000", 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1786999999999995e-05 2024-09-14T06:59:49.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40676 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 761856~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210659+0000", 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419753201, 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0085896489999999995, 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40676 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210659+0000", 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210659+0000", 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210660+0000", 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210662+0000", 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.796e-06 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210668+0000", 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2430000000000002e-06 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210673+0000", 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8099999999999997e-06 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219087+0000", 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084145279999999992 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219156+0000", 2024-09-14T06:59:49.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9277999999999996e-05 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219249+0000", 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2592999999999999e-05 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.673 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40677 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 688128~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210679+0000", 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4197336659999999, 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078670200000000006, 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40677 2024-09-14T06:59:49.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210679+0000", 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210679+0000", 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210679+0000", 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210681+0000", 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8160000000000001e-06 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210687+0000", 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0190000000000003e-06 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210692+0000", 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.977e-06 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.217747+0000", 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070545599999999997 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.217819+0000", 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2538000000000002e-05 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218546+0000", 2024-09-14T06:59:49.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072668600000000004 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40678 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 835584~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419713894, 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0087422799999999998, 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40678 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210701+0000", 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.81e-06 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210707+0000", 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0260000000000004e-06 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210712+0000", 2024-09-14T06:59:49.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219281+0000", 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008569587 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219351+0000", 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0055999999999995e-05 2024-09-14T06:59:49.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219441+0000", 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9608e-05 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40679 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 761856~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210718+0000", 2024-09-14T06:59:49.684 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4196947820000001, 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080154270000000003, 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40679 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210718+0000", 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210718+0000", 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210718+0000", 2024-09-14T06:59:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.08e-07 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210720+0000", 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.761e-06 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210726+0000", 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0959999999999997e-06 2024-09-14T06:59:49.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210731+0000", 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6160000000000004e-06 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218579+0000", 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078480240000000003 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218646+0000", 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7440000000000005e-05 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218733+0000", 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7082000000000004e-05 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40681 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 835584~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210737+0000", 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419675713, 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081938849999999997, 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40681 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210737+0000", 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210737+0000", 2024-09-14T06:59:49.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210737+0000", 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210739+0000", 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.685e-06 2024-09-14T06:59:49.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210745+0000", 2024-09-14T06:59:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7790000000000001e-06 2024-09-14T06:59:49.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210749+0000", 2024-09-14T06:59:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4859999999999999e-06 2024-09-14T06:59:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218765+0000", 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080161420000000004 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218841+0000", 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6114999999999999e-05 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218931+0000", 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9302000000000007e-05 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40705 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 393216~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4196546240000001, 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099565890000000001, 2024-09-14T06:59:49.698 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40705 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:49.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-14T06:59:49.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210760+0000", 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210768+0000", 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999997e-06 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210773+0000", 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0200000000000002e-06 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219470+0000", 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0086970339999999993 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219538+0000", 2024-09-14T06:59:49.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7550000000000002e-05 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220715+0000", 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001176564 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40706 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 466944~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.703 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4196326269999999, 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010139230000000001, 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40706 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210782+0000", 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210788+0000", 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1480000000000003e-06 2024-09-14T06:59:49.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210793+0000", 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220747+0000", 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099533769999999994 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220813+0000", 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5926999999999994e-05 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220919+0000", 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010641099999999999 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40718 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419612847, 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083164229999999999, 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40718 2024-09-14T06:59:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210802+0000", 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.886e-06 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210808+0000", 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0589999999999999e-06 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210815+0000", 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218961+0000", 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081464520000000002 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219029+0000", 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7178e-05 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219116+0000", 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7569000000000001e-05 2024-09-14T06:59:49.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40719 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 458752~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210822+0000", 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419590439, 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008481723, 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40719 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210822+0000", 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210822+0000", 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210823+0000", 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210824+0000", 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.796e-06 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210836+0000", 2024-09-14T06:59:49.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1567e-05 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210841+0000", 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3159999999999996e-06 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219153+0000", 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083116370000000002 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219220+0000", 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6958000000000005e-05 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219304+0000", 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4037999999999997e-05 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40739 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 679936~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.718 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210849+0000", 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4195633329999999, 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008625789, 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40739 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210849+0000", 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210849+0000", 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210850+0000", 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210852+0000", 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.153e-06 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210858+0000", 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4470000000000003e-06 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210863+0000", 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9490000000000004e-06 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219333+0000", 2024-09-14T06:59:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084702609999999998 2024-09-14T06:59:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219398+0000", 2024-09-14T06:59:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4968999999999994e-05 2024-09-14T06:59:49.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219475+0000", 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6625000000000006e-05 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40740 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 753664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210869+0000", 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4195431590000001, 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088293469999999995, 2024-09-14T06:59:49.723 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40740 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210869+0000", 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210869+0000", 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210870+0000", 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210886+0000", 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6568000000000001e-05 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210893+0000", 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8759999999999999e-06 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210899+0000", 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.648e-06 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219503+0000", 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0086043809999999995 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219612+0000", 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109064 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219699+0000", 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6419e-05 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.727 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40741 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 827392~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.419505689, 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0089720200000000007, 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40741 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.75e-07 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210909+0000", 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210916+0000", 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1659999999999999e-06 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210920+0000", 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8640000000000004e-06 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219728+0000", 2024-09-14T06:59:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088070700000000002 2024-09-14T06:59:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219794+0000", 2024-09-14T06:59:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6014999999999997e-05 2024-09-14T06:59:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219879+0000", 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5370000000000004e-05 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40760 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 360448~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.213533+0000", 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.416879327, 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075566080000000002, 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40760 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213533+0000", 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213533+0000", 2024-09-14T06:59:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213534+0000", 2024-09-14T06:59:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-14T06:59:49.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213539+0000", 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9740000000000001e-06 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213550+0000", 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0929e-05 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213562+0000", 2024-09-14T06:59:49.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1345000000000001e-05 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220947+0000", 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007385282 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221011+0000", 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3635999999999999e-05 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221090+0000", 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9220000000000004e-05 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40761 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.213575+0000", 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4168370850000001, 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077524830000000001, 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40761 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213575+0000", 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213575+0000", 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213576+0000", 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5899999999999996e-07 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213578+0000", 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.311e-06 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213585+0000", 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213590+0000", 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.989e-06 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221117+0000", 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075275630000000001 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221180+0000", 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2859000000000002e-05 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221328+0000", 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014778799999999999 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.742 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:49.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:59:49.745 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:49.744+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:49.845 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:49.844+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:49.946 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:49.946+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:49.960 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:51497 3.10 3.dcf8de30 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.960152+0000", 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.198e-05, 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7173e-05, 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 51497 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960152+0000", 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960152+0000", 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960153+0000", 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189999999999999e-06 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960157+0000", 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3209999999999998e-06 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960172+0000", 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4555e-05 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960180+0000", 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1419999999999993e-06 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:51498 3.10 3.dcf8de30 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.960197+0000", 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.6653000000000001e-05, 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7640999999999996e-05, 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 51498 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960197+0000", 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960197+0000", 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960197+0000", 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5300000000000004e-07 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960200+0000", 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.835e-06 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960209+0000", 2024-09-14T06:59:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3550000000000005e-06 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960215+0000", 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0959999999999997e-06 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:51499 3.10 3.dcf8de30 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.960223+0000", 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9999905, 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131693, 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 51499 2024-09-14T06:59:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960223+0000", 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960223+0000", 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960223+0000", 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960226+0000", 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2469999999999998e-06 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960233+0000", 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8739999999999997e-06 2024-09-14T06:59:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.960238+0000", 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3770000000000004e-06 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 3 2024-09-14T06:59:49.972 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:49.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:59:50.059 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:50.058+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:50.114 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-14T06:59:50.110+0000 7fbb91e53640 -1 mon.a@0(leader).osd e40 definitely_dead 0 2024-09-14T06:59:50.148 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:50.147+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:50.178 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:50.179 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:50.179 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:50.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:59:50.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:59:50.249 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:50.248+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:50.289 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-14T06:59:50.290 INFO:teuthology.orchestra.run.smithi007.stdout:1156 punch from 0xdd876ae to 0xdd9619b, (0xeaed bytes) 2024-09-14T06:59:50.305 INFO:teuthology.orchestra.run.smithi007.stdout:1157 trunc from 0xed445e7 to 0x5fa6324 2024-09-14T06:59:50.317 INFO:teuthology.orchestra.run.smithi007.stdout:1159 read 0x15ad864 thru 0x15ade11 (0x5ae bytes) 2024-09-14T06:59:50.325 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.6. 2024-09-14T06:59:50.329 INFO:teuthology.orchestra.run.smithi007.stdout:1161 writesame 0x7d56be4 thru 0x7d5801c data_size 0x1(0x1439 bytes) 2024-09-14T06:59:50.330 INFO:teuthology.orchestra.run.smithi007.stdout:1162 write 0x4363ba8 thru 0x436d72c (0x9b85 bytes) 2024-09-14T06:59:50.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:59:50.340 INFO:teuthology.orchestra.run.smithi007.stdout:1163 punch from 0x5f36117 to 0x5f3a3e5, (0x42ce bytes) 2024-09-14T06:59:50.340 INFO:teuthology.orchestra.run.smithi007.stdout:1164 write 0x18e7264 thru 0x18f38a3 (0xc640 bytes) 2024-09-14T06:59:50.366 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.630078359000002, 2024-09-14T06:59:50.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T06:59:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T06:59:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T06:59:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:59:50.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T06:59:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T06:59:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T06:59:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T06:59:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T06:59:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T06:59:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:50.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T06:59:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T06:59:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T06:59:50.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T06:59:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T06:59:50.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T06:59:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T06:59:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T06:59:50.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T06:59:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T06:59:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T06:59:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T06:59:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T06:59:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T06:59:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T06:59:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T06:59:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17618 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.236329+0000", 2024-09-14T06:59:50.382 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1274654210000001, 2024-09-14T06:59:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016649290000000001, 2024-09-14T06:59:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17618 2024-09-14T06:59:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T06:59:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236332+0000", 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237113+0000", 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078082900000000003 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237131+0000", 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7747000000000001e-05 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237141+0000", 2024-09-14T06:59:50.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0137e-05 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237221+0000", 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0482000000000006e-05 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237383+0000", 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016176100000000001 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237737+0000", 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035380999999999998 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.241638+0000", 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039011739999999999 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252854+0000", 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011216097 2024-09-14T06:59:50.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252919+0000", 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5129999999999995e-05 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252979+0000", 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9573e-05 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40284 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:9 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.204456+0000", 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.15933836, 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016005640000000002, 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40284 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204457+0000", 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204459+0000", 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-14T06:59:50.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204466+0000", 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8730000000000001e-06 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204472+0000", 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4779999999999999e-06 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207358+0000", 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028862549999999999 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207430+0000", 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2268000000000007e-05 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220462+0000", 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013031951999999999 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40646 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1581967259999999, 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017879991000000001, 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40646 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205605+0000", 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2260000000000002e-06 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205611+0000", 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3289999999999999e-06 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205622+0000", 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213726+0000", 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008104745 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213792+0000", 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5401999999999995e-05 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223478+0000", 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096862100000000007 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40683 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1581248070000001, 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016494009, 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40683 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.401 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205672+0000", 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0379999999999998e-06 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205679+0000", 2024-09-14T06:59:50.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-14T06:59:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205684+0000", 2024-09-14T06:59:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9359999999999998e-06 2024-09-14T06:59:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221845+0000", 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016161422000000002 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221913+0000", 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7669000000000006e-05 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222164+0000", 2024-09-14T06:59:50.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025130600000000002 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40685 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205690+0000", 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1581044029999998, 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016657634000000001, 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40685 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205691+0000", 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205693+0000", 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-14T06:59:50.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205699+0000", 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205703+0000", 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5240000000000002e-06 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222190+0000", 2024-09-14T06:59:50.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016486752 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222257+0000", 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7095999999999996e-05 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222348+0000", 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1061000000000001e-05 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40687 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1580851729999999, 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016826543999999999, 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40687 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205712+0000", 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205718+0000", 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-14T06:59:50.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205723+0000", 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222374+0000", 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016650933999999999 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222445+0000", 2024-09-14T06:59:50.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1304999999999996e-05 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222536+0000", 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1104000000000001e-05 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40720 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205826+0000", 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1579685770000001, 2024-09-14T06:59:50.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016893371000000001, 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40720 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T06:59:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205827+0000", 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.58e-07 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205834+0000", 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2729999999999997e-06 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205841+0000", 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8260000000000004e-06 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205846+0000", 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1769999999999997e-06 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222562+0000", 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016716178000000002 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222633+0000", 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0461000000000002e-05 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222720+0000", 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6997999999999996e-05 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40721 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.157940875, 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044822000000001, 2024-09-14T06:59:50.421 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40721 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T06:59:50.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205857+0000", 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-06 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205863+0000", 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205868+0000", 2024-09-14T06:59:50.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222745+0000", 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016877293000000002 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222814+0000", 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8806000000000005e-05 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222899+0000", 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4913000000000005e-05 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40742 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1579203090000001, 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017201659000000001, 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40742 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:50.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205892+0000", 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7008e-05 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205899+0000", 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1489999999999997e-06 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205904+0000", 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222923+0000", 2024-09-14T06:59:50.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017018229999999999 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222991+0000", 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8545000000000002e-05 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223076+0000", 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4938999999999994e-05 2024-09-14T06:59:50.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40743 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205913+0000", 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.1578815769999999, 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343654999999999, 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40743 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T06:59:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205914+0000", 2024-09-14T06:59:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-07 2024-09-14T06:59:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205916+0000", 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0559999999999999e-06 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205922+0000", 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3659999999999997e-06 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205927+0000", 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8169999999999998e-06 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223101+0000", 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017174452 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223168+0000", 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6756999999999998e-05 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223257+0000", 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8700000000000001e-05 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43363 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.91070781099999998, 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015032403, 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43363 2024-09-14T06:59:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453089+0000", 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9980000000000002e-06 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453096+0000", 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453101+0000", 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8960000000000002e-06 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465082+0000", 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011981083 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465162+0000", 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0588999999999997e-05 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468119+0000", 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002957079 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43452 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.91044012100000005, 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014487193000000001, 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43452 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3299999999999998e-07 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453357+0000", 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8500000000000001e-06 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453363+0000", 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3239999999999999e-06 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453368+0000", 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2290000000000002e-06 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467645+0000", 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014276812999999999 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467720+0000", 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4492000000000004e-05 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467842+0000", 2024-09-14T06:59:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012215199999999999 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43453 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 78848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.91041991, 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015012878, 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43453 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453377+0000", 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1050000000000002e-06 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453384+0000", 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453389+0000", 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-06 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467869+0000", 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014479845999999999 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467944+0000", 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5680999999999996e-05 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468388+0000", 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000443473 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43576 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 99328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.90798704500000005, 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014560714000000001, 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43576 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455809+0000", 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.091e-06 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455812+0000", 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199999999999999e-06 2024-09-14T06:59:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455815+0000", 2024-09-14T06:59:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.537e-06 2024-09-14T06:59:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470167+0000", 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014352746 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470245+0000", 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7730000000000003e-05 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470368+0000", 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012338 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43577 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 102400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.90797696400000005, 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014789722999999999, 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43577 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455820+0000", 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4270000000000001e-06 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455823+0000", 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0139999999999999e-06 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455825+0000", 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4980000000000001e-06 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470402+0000", 2024-09-14T06:59:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014576858999999999 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470485+0000", 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2731000000000005e-05 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470608+0000", 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012297299999999999 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43578 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.90795630800000005, 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015008412, 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43578 2024-09-14T06:59:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4400000000000001e-07 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455841+0000", 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2749999999999998e-06 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455848+0000", 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9349999999999997e-06 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455854+0000", 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-06 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470639+0000", 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014785382999999999 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470716+0000", 2024-09-14T06:59:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7155000000000002e-05 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470847+0000", 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013058000000000001 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43579 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.90793294800000002, 2024-09-14T06:59:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01522916, 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43579 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455864+0000", 2024-09-14T06:59:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455870+0000", 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455875+0000", 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9509999999999997e-06 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470879+0000", 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015003729 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470960+0000", 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1228000000000002e-05 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471091+0000", 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131018 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43580 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 110592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455881+0000", 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.90791341000000003, 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015447047, 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43580 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455882+0000", 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455884+0000", 2024-09-14T06:59:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2010000000000002e-06 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455891+0000", 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5690000000000001e-06 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455896+0000", 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3639999999999998e-06 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471123+0000", 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015226805 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471200+0000", 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7405999999999997e-05 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471328+0000", 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012835099999999999 2024-09-14T06:59:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43581 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455903+0000", 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.90789150900000004, 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015668949000000001, 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43581 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455904+0000", 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455910+0000", 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6619999999999999e-06 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455917+0000", 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5769999999999999e-06 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455923+0000", 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6779999999999997e-06 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471360+0000", 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015436854999999999 2024-09-14T06:59:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471443+0000", 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3815999999999998e-05 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471572+0000", 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012892900000000001 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:50.480 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:50.480 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:50.365+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:50.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:59:50.482 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:50.482 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:50.482 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:50.483 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:50.483 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:50.483 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:50.484 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:50.484 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:50.485 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:50.485 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:50.485 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:50.486 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:50.497 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:50.498 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:50.496+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:50.514 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:50.514 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:50.586 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:50.585+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:50.606 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:50.607 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:50.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:59:50.687 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:50.687+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:50.763 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:50.763 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:50.772 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:59:50.816 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:50.815+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:50.879 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:50.879 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:50.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:59:50.929 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:50.928+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:50.990 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:50.989+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:51.036 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:51.036 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:51.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:59:51.129 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:51.128+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:51.191 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:51.191 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:51.192 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:51.190+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:51.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:59:51.293 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:51.292+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:51.326 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T06:59:51.362 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:51.362 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:51.372 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:59:51.416 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:51.415+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:51.483 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:51.483 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:51.491 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:59:51.494 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:51.493+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:51.594 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:51.594+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:51.635 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:51.636 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:51.644 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:59:51.695 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:51.695+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:51.771 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:51.772 INFO:teuthology.orchestra.run.smithi007.stdout:1166 write 0x29ff8a3 thru 0x2a069aa (0x7108 bytes) 2024-09-14T06:59:51.786 INFO:teuthology.orchestra.run.smithi007.stdout:1168 writesame 0xaf50988 thru 0xaf53110 data_size 0x1(0x2789 bytes) 2024-09-14T06:59:51.793 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:51.793 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:51.796 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:51.795+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:51.801 INFO:teuthology.orchestra.run.smithi007.stdout:1169 trunc from 0xaf53111 to 0xd84242 2024-09-14T06:59:51.802 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:59:51.898 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:51.897+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:51.973 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:51.973 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:51.983 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:59:51.999 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:51.998+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:52.100 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:52.099+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:52.123 INFO:teuthology.orchestra.run.smithi007.stdout:1170 write 0xba31149 thru 0xba3e344 (0xd1fc bytes) 2024-09-14T06:59:52.126 INFO:teuthology.orchestra.run.smithi007.stdout:1171 trunc from 0xba3e345 to 0xc74d612 2024-09-14T06:59:52.133 INFO:teuthology.orchestra.run.smithi007.stdout:1172 read 0x7837798 thru 0x7840f3f (0x97a8 bytes) 2024-09-14T06:59:52.133 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:52.133 INFO:teuthology.orchestra.run.smithi007.stdout:1174 read 0xb5367b1 thru 0xb539ef9 (0x3749 bytes) 2024-09-14T06:59:52.133 INFO:teuthology.orchestra.run.smithi007.stdout:1175 read 0x5164d25 thru 0x516a797 (0x5a73 bytes) 2024-09-14T06:59:52.133 INFO:teuthology.orchestra.run.smithi007.stdout:1177 punch from 0x5923f1d to 0x59263a2, (0x2485 bytes) 2024-09-14T06:59:52.133 INFO:teuthology.orchestra.run.smithi007.stdout:1178 read 0x46e03c3 thru 0x46e29ba (0x25f8 bytes) 2024-09-14T06:59:52.133 INFO:teuthology.orchestra.run.smithi007.stdout:1179 punch from 0x521f144 to 0x52266c9, (0x7585 bytes) 2024-09-14T06:59:52.133 INFO:teuthology.orchestra.run.smithi007.stdout:1180 read 0xabff017 thru 0xac08419 (0x9403 bytes) 2024-09-14T06:59:52.133 INFO:teuthology.orchestra.run.smithi007.stdout:1183 punch from 0x2614d5d to 0x26166cd, (0x1970 bytes) 2024-09-14T06:59:52.134 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:52.134 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:52.138 INFO:teuthology.orchestra.run.smithi007.stdout:1185 trunc from 0xc74d612 to 0x7283542 2024-09-14T06:59:52.143 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:59:52.154 INFO:teuthology.orchestra.run.smithi007.stdout:1187 trunc from 0x7283542 to 0x22b6b39 2024-09-14T06:59:52.172 INFO:teuthology.orchestra.run.smithi007.stdout:1188 trunc from 0x22b6b39 to 0xc3cfeff 2024-09-14T06:59:52.178 INFO:teuthology.orchestra.run.smithi007.stdout:1189 write 0x6b678b7 thru 0x6b7578c (0xded6 bytes) 2024-09-14T06:59:52.181 INFO:teuthology.orchestra.run.smithi007.stdout:1190 write 0x9bad9c9 thru 0x9bb9cc1 (0xc2f9 bytes) 2024-09-14T06:59:52.183 INFO:teuthology.orchestra.run.smithi007.stdout:1191 read 0x265532d thru 0x265b6a9 (0x637d bytes) 2024-09-14T06:59:52.183 INFO:teuthology.orchestra.run.smithi007.stdout:1192 clone 11 order 19 su 16384 sc 9 2024-09-14T06:59:52.201 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:52.201+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:52.287 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:52.288 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:52.296 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:59:52.340 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:52.339+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:52.402 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone10 from 0xc3cfeff (overlap 0xd84242) to 0x15a039 2024-09-14T06:59:52.403 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:52.402+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:52.403 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:52.403 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:52.413 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:59:52.497 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-14T06:59:52.504 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:52.503+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:52.513 INFO:teuthology.orchestra.run.smithi007.stdout:1193 writesame 0x600285f thru 0x600c40f data_size 0x1(0x9bb1 bytes) 2024-09-14T06:59:52.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:59:52.529 INFO:teuthology.orchestra.run.smithi007.stdout:1194 writesame 0x1bc6cf8 thru 0x1bcf4d9 data_size 0x1(0x87e2 bytes) 2024-09-14T06:59:52.543 INFO:teuthology.orchestra.run.smithi007.stdout:1196 read 0xb45afcd thru 0xb4671c8 (0xc1fc bytes) 2024-09-14T06:59:52.543 INFO:teuthology.orchestra.run.smithi007.stdout:1197 punch from 0x87af8f7 to 0x87b9118, (0x9821 bytes) 2024-09-14T06:59:52.544 INFO:teuthology.orchestra.run.smithi007.stdout:1198 trunc from 0xc3cfeff to 0xd69a09d 2024-09-14T06:59:52.555 INFO:teuthology.orchestra.run.smithi007.stdout:1200 trunc from 0xd69a09d to 0x89e1749 2024-09-14T06:59:52.561 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:52.561 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:52.571 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:59:52.577 INFO:teuthology.orchestra.run.smithi007.stdout:1201 write 0x9901b4f thru 0x990cdc5 (0xb277 bytes) 2024-09-14T06:59:52.584 INFO:teuthology.orchestra.run.smithi007.stdout:1202 write 0x41425f8 thru 0x41457d5 (0x31de bytes) 2024-09-14T06:59:52.590 INFO:teuthology.orchestra.run.smithi007.stdout:1203 read 0x7a73a19 thru 0x7a7b421 (0x7a09 bytes) 2024-09-14T06:59:52.590 INFO:teuthology.orchestra.run.smithi007.stdout:1204 write 0x98eb450 thru 0x98f3342 (0x7ef3 bytes) 2024-09-14T06:59:52.601 INFO:teuthology.orchestra.run.smithi007.stdout:1205 write 0x9e47756 thru 0x9e543a0 (0xcc4b bytes) 2024-09-14T06:59:52.605 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:52.604+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:52.608 INFO:teuthology.orchestra.run.smithi007.stdout:1206 read 0x7a58925 thru 0x7a60fe2 (0x86be bytes) 2024-09-14T06:59:52.610 INFO:teuthology.orchestra.run.smithi007.stdout:1207 trunc from 0x9e543a1 to 0x818dc94 2024-09-14T06:59:52.635 INFO:teuthology.orchestra.run.smithi007.stdout:1209 punch from 0x8f1355 to 0x8f1577, (0x222 bytes) 2024-09-14T06:59:52.644 INFO:teuthology.orchestra.run.smithi007.stdout:1210 writesame 0xa962d78 thru 0xa9683b6 data_size 0x1(0x563f bytes) 2024-09-14T06:59:52.644 INFO:teuthology.orchestra.run.smithi007.stdout:1211 read 0x8d1235a thru 0x8d17cc7 (0x596e bytes) 2024-09-14T06:59:52.644 INFO:teuthology.orchestra.run.smithi007.stdout:1212 write 0x27d0f2b thru 0x27d6a1d (0x5af3 bytes) 2024-09-14T06:59:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:1213 read 0x4d83332 thru 0x4d919d6 (0xe6a5 bytes) 2024-09-14T06:59:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:1214 read 0x173c874 thru 0x17497e8 (0xcf75 bytes) 2024-09-14T06:59:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:1215 write 0x31b5daa thru 0x31b9390 (0x35e7 bytes) 2024-09-14T06:59:52.655 INFO:teuthology.orchestra.run.smithi007.stdout:1216 writesame 0x57b8e76 thru 0x57c7c46 data_size 0x1(0xedd1 bytes) 2024-09-14T06:59:52.676 INFO:teuthology.orchestra.run.smithi007.stdout:1218 read 0x24cfd59 thru 0x24db34c (0xb5f4 bytes) 2024-09-14T06:59:52.677 INFO:teuthology.orchestra.run.smithi007.stdout:1219 writesame 0x89a2339 thru 0x89ae78a data_size 0xd(0xc452 bytes) 2024-09-14T06:59:52.680 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:52.681 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:52.684 INFO:teuthology.orchestra.run.smithi007.stdout:1220 punch from 0x2aef34d to 0x2afbede, (0xcb91 bytes) 2024-09-14T06:59:52.688 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:52.691 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:59:52.691 INFO:teuthology.orchestra.run.smithi007.stdout:1222 trunc from 0xa9683b7 to 0x49342eb 2024-09-14T06:59:52.706 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:52.705+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:52.713 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:52.713 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:52.720 INFO:teuthology.orchestra.run.smithi007.stdout:1224 write 0x214891f thru 0x214f028 (0x670a bytes) 2024-09-14T06:59:52.724 INFO:teuthology.orchestra.run.smithi007.stdout:1225 read 0x4376ba8 thru 0x4378140 (0x1599 bytes) 2024-09-14T06:59:52.724 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:52.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:52.739 INFO:teuthology.orchestra.run.smithi007.stdout:1228 write 0xdf96565 thru 0xdf97bda (0x1676 bytes) 2024-09-14T06:59:52.743 INFO:teuthology.orchestra.run.smithi007.stdout:1230 punch from 0x18da9d4 to 0x18e4851, (0x9e7d bytes) 2024-09-14T06:59:52.743 INFO:teuthology.orchestra.run.smithi007.stdout:1231 read 0x87d1be1 thru 0x87d5c8d (0x40ad bytes) 2024-09-14T06:59:52.743 INFO:teuthology.orchestra.run.smithi007.stdout:1232 read 0x3422ebb thru 0x34297d9 (0x691f bytes) 2024-09-14T06:59:52.752 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:52.755 INFO:teuthology.orchestra.run.smithi007.stdout:1233 trunc from 0xdf97bdb to 0x3a0c0b9 2024-09-14T06:59:52.782 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:52.782 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:52.799 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:52.799 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:52.802 INFO:teuthology.orchestra.run.smithi007.stdout:1236 read 0x17a04aa thru 0x17a428e (0x3de5 bytes) 2024-09-14T06:59:52.806 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:52.806+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:52.807 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:52.807 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:52.809 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:59:52.809 INFO:teuthology.orchestra.run.smithi007.stdout:1237 writesame 0x49e542e thru 0x49eb641 data_size 0x2(0x6214 bytes) 2024-09-14T06:59:52.809 INFO:teuthology.orchestra.run.smithi007.stdout:1238 write 0x40dce8c thru 0x40e5a8b (0x8c00 bytes) 2024-09-14T06:59:52.826 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:52.827 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:52.830 INFO:teuthology.orchestra.run.smithi007.stdout:1239 writesame 0xebbd398 thru 0xebc13c0 data_size 0x49(0x4029 bytes) 2024-09-14T06:59:52.830 INFO:teuthology.orchestra.run.smithi007.stdout:1240 read 0xb6f890e thru 0xb7051a7 (0xc89a bytes) 2024-09-14T06:59:52.845 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:52.846 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:52.846 INFO:teuthology.orchestra.run.smithi007.stdout:1246 trunc from 0xebc13c1 to 0x11be70e 2024-09-14T06:59:52.902 INFO:teuthology.orchestra.run.smithi007.stdout:1248 write 0xb8a2ae2 thru 0xb8a999c (0x6ebb bytes) 2024-09-14T06:59:52.908 INFO:teuthology.orchestra.run.smithi007.stdout:1249 read 0x84f7e5b thru 0x84f85d3 (0x779 bytes) 2024-09-14T06:59:52.908 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:52.908 INFO:teuthology.orchestra.run.smithi007.stdout:1251 read 0x4ddb119 thru 0x4de4e82 (0x9d6a bytes) 2024-09-14T06:59:52.908 INFO:teuthology.orchestra.run.smithi007.stdout:1253 read 0x5f7b527 thru 0x5f87b54 (0xc62e bytes) 2024-09-14T06:59:52.908 INFO:teuthology.orchestra.run.smithi007.stdout:1255 read 0x97f4140 thru 0x98038c4 (0xf785 bytes) 2024-09-14T06:59:52.908 INFO:teuthology.orchestra.run.smithi007.stdout:1256 punch from 0x1b9159 to 0x1c7994, (0xe83b bytes) 2024-09-14T06:59:52.908 INFO:teuthology.orchestra.run.smithi007.stdout:1257 read 0x5cb8063 thru 0x5cc718b (0xf129 bytes) 2024-09-14T06:59:52.908 INFO:teuthology.orchestra.run.smithi007.stdout:1258 write 0x9db071 thru 0x9e8f20 (0xdeb0 bytes) 2024-09-14T06:59:52.908 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:52.907+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:52.924 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:52.924 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:52.924 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:52.924 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:52.924 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:52.930 INFO:teuthology.orchestra.run.smithi007.stdout:1263 writesame 0x3c9318c thru 0x3c9b0e5 data_size 0x1(0x7f5a bytes) 2024-09-14T06:59:52.933 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:59:52.943 INFO:teuthology.orchestra.run.smithi007.stdout:1264 trunc from 0xb8a999d to 0xe1c353e 2024-09-14T06:59:52.962 INFO:teuthology.orchestra.run.smithi007.stdout:1265 trunc from 0xe1c353e to 0x1ff077d 2024-09-14T06:59:52.980 INFO:teuthology.orchestra.run.smithi007.stdout:1268 writesame 0x5bdcff3 thru 0x5be6ebd data_size 0x35(0x9ecb bytes) 2024-09-14T06:59:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:1269 read 0x3cce00 thru 0x3cfd04 (0x2f05 bytes) 2024-09-14T06:59:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:52.981 INFO:teuthology.orchestra.run.smithi007.stdout:1273 writesame 0x3f0a3d0 thru 0x3f187ff data_size 0x18(0xe430 bytes) 2024-09-14T06:59:52.993 INFO:teuthology.orchestra.run.smithi007.stdout:1274 writesame 0x6a28c6f thru 0x6a35b7c data_size 0x11(0xcf0e bytes) 2024-09-14T06:59:52.993 INFO:teuthology.orchestra.run.smithi007.stdout:1275 write 0x153eeab thru 0x1545152 (0x62a8 bytes) 2024-09-14T06:59:53.004 INFO:teuthology.orchestra.run.smithi007.stdout:1276 trunc from 0x6a35b7d to 0x15eb412 2024-09-14T06:59:53.008 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:53.007+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:53.014 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:53.020 INFO:teuthology.orchestra.run.smithi007.stdout:1278 writesame 0x2b0524b thru 0x2b121d7 data_size 0x3(0xcf8d bytes) 2024-09-14T06:59:53.029 INFO:teuthology.orchestra.run.smithi007.stdout:1279 write 0x5a5971a thru 0x5a65c86 (0xc56d bytes) 2024-09-14T06:59:53.040 INFO:teuthology.orchestra.run.smithi007.stdout:1280 trunc from 0x5a65c87 to 0xb450f37 2024-09-14T06:59:53.045 INFO:teuthology.orchestra.run.smithi007.stdout:1281 punch from 0x49b313e to 0x49b6b4e, (0x3a10 bytes) 2024-09-14T06:59:53.046 INFO:teuthology.orchestra.run.smithi007.stdout:1283 read 0x53276f3 thru 0x532cb01 (0x540f bytes) 2024-09-14T06:59:53.046 INFO:teuthology.orchestra.run.smithi007.stdout:1284 read 0xa83aecf thru 0xa83f113 (0x4245 bytes) 2024-09-14T06:59:53.046 INFO:teuthology.orchestra.run.smithi007.stdout:1286 write 0x59326f9 thru 0x593cb96 (0xa49e bytes) 2024-09-14T06:59:53.050 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:53.051 INFO:teuthology.orchestra.run.smithi007.stdout:1288 punch from 0x4d63730 to 0x4d734d8, (0xfda8 bytes) 2024-09-14T06:59:53.051 INFO:teuthology.orchestra.run.smithi007.stdout:1289 read 0x211854e thru 0x2126c07 (0xe6ba bytes) 2024-09-14T06:59:53.051 INFO:teuthology.orchestra.run.smithi007.stdout:1290 writesame 0x744f39a thru 0x7455687 data_size 0x36(0x62ee bytes) 2024-09-14T06:59:53.055 INFO:teuthology.orchestra.run.smithi007.stdout:1292 read 0x39f397e thru 0x3a00700 (0xcd83 bytes) 2024-09-14T06:59:53.055 INFO:teuthology.orchestra.run.smithi007.stdout:1293 punch from 0x3cee2d5 to 0x3cf92aa, (0xafd5 bytes) 2024-09-14T06:59:53.055 INFO:teuthology.orchestra.run.smithi007.stdout:1294 write 0x5ed295a thru 0x5ed6789 (0x3e30 bytes) 2024-09-14T06:59:53.058 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:53.058 INFO:teuthology.orchestra.run.smithi007.stdout:1296 read 0x94d853 thru 0x95a643 (0xcdf1 bytes) 2024-09-14T06:59:53.058 INFO:teuthology.orchestra.run.smithi007.stdout:1297 write 0x5c39c30 thru 0x5c3e52b (0x48fc bytes) 2024-09-14T06:59:53.061 INFO:teuthology.orchestra.run.smithi007.stdout:1299 punch from 0x955d792 to 0x956b22f, (0xda9d bytes) 2024-09-14T06:59:53.062 INFO:teuthology.orchestra.run.smithi007.stdout:1301 punch from 0x63d9903 to 0x63e38a4, (0x9fa1 bytes) 2024-09-14T06:59:53.062 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:53.062 INFO:teuthology.orchestra.run.smithi007.stdout:1304 write 0x6f5a05d thru 0x6f68851 (0xe7f5 bytes) 2024-09-14T06:59:53.068 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:53.069 INFO:teuthology.orchestra.run.smithi007.stdout:1306 read 0xa9d5fb2 thru 0xa9d907c (0x30cb bytes) 2024-09-14T06:59:53.069 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:53.070 INFO:teuthology.orchestra.run.smithi007.stdout:1308 trunc from 0xb450f37 to 0xcfdf65d 2024-09-14T06:59:53.075 INFO:teuthology.orchestra.run.smithi007.stdout:1309 read 0x49aaae3 thru 0x49ba7b8 (0xfcd6 bytes) 2024-09-14T06:59:53.075 INFO:teuthology.orchestra.run.smithi007.stdout:1310 write 0x77a22d0 thru 0x77a508e (0x2dbf bytes) 2024-09-14T06:59:53.081 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8489 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8208384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670275+0000", 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4100203409999992, 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033127139, 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.082 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8489 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670276+0000", 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670277+0000", 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6109999999999999e-06 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670283+0000", 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7409999999999998e-06 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670287+0000", 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1679999999999997e-06 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703311+0000", 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033023640999999999 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703352+0000", 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1168999999999998e-05 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703402+0000", 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0424999999999999e-05 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8490 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.086 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4100027579999992, 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033303263, 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8490 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:53.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7800000000000002e-07 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670295+0000", 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670300+0000", 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5369999999999997e-06 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670304+0000", 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8789999999999996e-06 2024-09-14T06:59:53.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703433+0000", 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033128697999999998 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703510+0000", 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6662000000000006e-05 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703596+0000", 2024-09-14T06:59:53.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6614999999999997e-05 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8491 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8454144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4099851280000006, 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033389611999999999, 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.091 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8491 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670317+0000", 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.19e-06 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670323+0000", 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670331+0000", 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1280000000000008e-06 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703611+0000", 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033279557000000001 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703650+0000", 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9410999999999999e-05 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703700+0000", 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0117000000000001e-05 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8492 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8568832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.095 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4099570779999997, 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033561793999999999, 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8492 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670341+0000", 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8700000000000001e-06 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670347+0000", 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670351+0000", 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4660000000000001e-06 2024-09-14T06:59:53.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703731+0000", 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033379502999999998 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703812+0000", 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1192000000000005e-05 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703900+0000", 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8382999999999997e-05 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8493 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8683520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4099385800000004, 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033663516999999997, 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.100 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8493 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670359+0000", 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.685e-06 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670365+0000", 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.57e-06 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670369+0000", 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4460000000000003e-06 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703928+0000", 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033558530000000003 2024-09-14T06:59:53.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703970+0000", 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1909999999999997e-05 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704021+0000", 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0988e-05 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8494 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8798208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4099089839999994, 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033828579999999997, 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8494 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670389+0000", 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.64e-06 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670393+0000", 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670396+0000", 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6369999999999999e-06 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704051+0000", 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033655293000000003 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704128+0000", 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7453000000000005e-05 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704215+0000", 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6884000000000004e-05 2024-09-14T06:59:53.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8495 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8912896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670400+0000", 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4098954339999992, 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033930725000000002, 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8495 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670401+0000", 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.15e-07 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670402+0000", 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.850000000000001e-07 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670405+0000", 2024-09-14T06:59:53.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.225e-06 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670407+0000", 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.34e-06 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704240+0000", 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033833019999999998 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704281+0000", 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1332999999999999e-05 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704331+0000", 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9506999999999999e-05 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8496 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9027584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.113 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4098841100000001, 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034114617, 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8496 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670413+0000", 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8999999999999995e-07 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670416+0000", 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2729999999999998e-06 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670418+0000", 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3709999999999998e-06 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704362+0000", 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033943917999999997 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704439+0000", 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6922999999999995e-05 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704526+0000", 2024-09-14T06:59:53.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7035999999999998e-05 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8497 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9142272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670421+0000", 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4098742219999991, 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034220315000000001, 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.118 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8497 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T06:59:53.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670422+0000", 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670426+0000", 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5029999999999999e-06 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670430+0000", 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670432+0000", 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4269999999999998e-06 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704552+0000", 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034120277999999997 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704592+0000", 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9884e-05 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704642+0000", 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9623999999999998e-05 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8498 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9256960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:53.122 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4098604590000008, 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034407434000000001, 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8498 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:53.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000001e-07 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670436+0000", 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999996e-07 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670439+0000", 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670442+0000", 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3120000000000001e-06 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704673+0000", 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034231175000000003 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704749+0000", 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6069999999999995e-05 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704843+0000", 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3622000000000006e-05 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.126 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8499 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9371648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.409850939, 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034523731000000002, 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.127 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8499 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670446+0000", 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670449+0000", 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809999999999998e-06 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670455+0000", 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7930000000000003e-06 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704869+0000", 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034413931000000002 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704910+0000", 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1032999999999998e-05 2024-09-14T06:59:53.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704968+0000", 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8897999999999998e-05 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8500 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9486336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4098376619999993, 2024-09-14T06:59:53.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034708731, 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8500 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670459+0000", 2024-09-14T06:59:53.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2999999999999999e-07 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670462+0000", 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1350000000000001e-06 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670470+0000", 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7400000000000004e-06 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705002+0000", 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034531470000000002 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705079+0000", 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7472999999999995e-05 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705167+0000", 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7750000000000005e-05 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.135 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8501 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9601024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670476+0000", 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4098192330000003, 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034806278000000003, 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8501 2024-09-14T06:59:53.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670477+0000", 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.44e-07 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670479+0000", 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7740000000000001e-06 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670485+0000", 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9150000000000001e-06 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670489+0000", 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5469999999999998e-06 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705191+0000", 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034701896000000003 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705234+0000", 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2394000000000001e-05 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705283+0000", 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9208e-05 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8502 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9715712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4097988939999997, 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034979270999999999, 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.140 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8502 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670506+0000", 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9740000000000008e-06 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670512+0000", 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670517+0000", 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7049999999999998e-06 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705313+0000", 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034795858999999998 2024-09-14T06:59:53.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705391+0000", 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8166999999999999e-05 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705476+0000", 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5285999999999996e-05 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8503 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9830400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670523+0000", 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4097724720000002, 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053094, 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8503 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.145 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670524+0000", 2024-09-14T06:59:53.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670525+0000", 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.581e-06 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670531+0000", 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8969999999999996e-06 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670536+0000", 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4630000000000003e-06 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705488+0000", 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034952451000000002 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705526+0000", 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8396999999999998e-05 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705576+0000", 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9926000000000003e-05 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8504 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9945088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4097541339999999, 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035228601999999998, 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.149 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8504 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:53.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670543+0000", 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5820000000000001e-06 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670549+0000", 2024-09-14T06:59:53.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670554+0000", 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3699999999999997e-06 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705607+0000", 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053594 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705685+0000", 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8102999999999994e-05 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705770+0000", 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4678000000000005e-05 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8505 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10059776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.153 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670559+0000", 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4097363470000008, 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035318548999999998, 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8505 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670560+0000", 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3599999999999999e-07 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670561+0000", 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670568+0000", 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9789999999999996e-06 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670572+0000", 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2509999999999996e-06 2024-09-14T06:59:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705784+0000", 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035211724 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705827+0000", 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3276999999999999e-05 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705878+0000", 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0488000000000002e-05 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8506 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10174464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670579+0000", 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4097162609999998, 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035495512999999999, 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.158 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8506 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670580+0000", 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670582+0000", 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670587+0000", 2024-09-14T06:59:53.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.767e-06 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670592+0000", 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2819999999999998e-06 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705909+0000", 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.03531753 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705987+0000", 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706075+0000", 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7998999999999996e-05 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8507 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10289152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670597+0000", 2024-09-14T06:59:53.162 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4096984349999993, 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035582134000000001, 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8507 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T06:59:53.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670598+0000", 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9900000000000002e-07 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670599+0000", 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6029999999999999e-06 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670609+0000", 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.004e-05 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670614+0000", 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0749999999999997e-06 2024-09-14T06:59:53.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706089+0000", 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035474358999999997 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706129+0000", 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0204999999999999e-05 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706179+0000", 2024-09-14T06:59:53.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0553000000000002e-05 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8508 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10403840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.4096748059999999, 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035743559000000001, 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:53.167 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8508 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:53.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2800000000000002e-07 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670623+0000", 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670629+0000", 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.02e-06 2024-09-14T06:59:53.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670634+0000", 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6280000000000004e-06 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706200+0000", 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035566564000000002 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:53.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706276+0000", 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5801000000000002e-05 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706364+0000", 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8176000000000004e-05 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:53.171 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:53.173 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:53.172+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:53.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:59:53.175 INFO:teuthology.orchestra.run.smithi007.stdout:1311 trunc from 0xcfdf65d to 0x43c1e97 2024-09-14T06:59:53.175 INFO:teuthology.orchestra.run.smithi007.stdout:1312 read 0x30cca61 thru 0x30d2b77 (0x6117 bytes) 2024-09-14T06:59:53.175 INFO:teuthology.orchestra.run.smithi007.stdout:1314 write 0xd78a435 thru 0xd7954ab (0xb077 bytes) 2024-09-14T06:59:53.175 INFO:teuthology.orchestra.run.smithi007.stdout:1316 trunc from 0xd7954ac to 0xa6d1324 2024-09-14T06:59:53.175 INFO:teuthology.orchestra.run.smithi007.stdout:1317 punch from 0xa141091 to 0xa145d36, (0x4ca5 bytes) 2024-09-14T06:59:53.175 INFO:teuthology.orchestra.run.smithi007.stdout:1319 writesame 0xde523f9 thru 0xde58ee0 data_size 0x8(0x6ae8 bytes) 2024-09-14T06:59:53.175 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:53.175 INFO:teuthology.orchestra.run.smithi007.stdout:1322 write 0x47be042 thru 0x47c586f (0x782e bytes) 2024-09-14T06:59:53.175 INFO:teuthology.orchestra.run.smithi007.stdout:1323 punch from 0x390e4f0 to 0x391349c, (0x4fac bytes) 2024-09-14T06:59:53.176 INFO:teuthology.orchestra.run.smithi007.stdout:1324 punch from 0x2633851 to 0x264198d, (0xe13c bytes) 2024-09-14T06:59:53.176 INFO:teuthology.orchestra.run.smithi007.stdout:1325 read 0x8f286bb thru 0x8f339ec (0xb332 bytes) 2024-09-14T06:59:53.176 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:53.176 INFO:teuthology.orchestra.run.smithi007.stdout:1329 write 0xb1a85c4 thru 0xb1b460b (0xc048 bytes) 2024-09-14T06:59:53.176 INFO:teuthology.orchestra.run.smithi007.stdout:1331 writesame 0x491124f thru 0x491bbaf data_size 0x1(0xa961 bytes) 2024-09-14T06:59:53.176 INFO:teuthology.orchestra.run.smithi007.stdout:1332 writesame 0x5fd4695 thru 0x5fe2791 data_size 0x3(0xe0fd bytes) 2024-09-14T06:59:53.178 INFO:teuthology.orchestra.run.smithi007.stdout:1333 write 0x4958c1c thru 0x496004c (0x7431 bytes) 2024-09-14T06:59:53.179 INFO:teuthology.orchestra.run.smithi007.stdout:1334 writesame 0x8437606 thru 0x843edd3 data_size 0x5(0x77ce bytes) 2024-09-14T06:59:53.186 INFO:teuthology.orchestra.run.smithi007.stdout:1336 punch from 0xc1b4279 to 0xc1b59bf, (0x1746 bytes) 2024-09-14T06:59:53.195 INFO:teuthology.orchestra.run.smithi007.stdout:1338 trunc from 0xde58ee1 to 0x63633b2 2024-09-14T06:59:53.210 INFO:teuthology.orchestra.run.smithi007.stdout:1340 trunc from 0x63633b2 to 0x9c131c0 2024-09-14T06:59:53.222 INFO:teuthology.orchestra.run.smithi007.stdout:1342 trunc from 0x9c131c0 to 0x37aad1a 2024-09-14T06:59:53.240 INFO:teuthology.orchestra.run.smithi007.stdout:1343 write 0x494fc89 thru 0x495f42f (0xf7a7 bytes) 2024-09-14T06:59:53.246 INFO:teuthology.orchestra.run.smithi007.stdout:1344 punch from 0x1665527 to 0x166c6ce, (0x71a7 bytes) 2024-09-14T06:59:53.247 INFO:teuthology.orchestra.run.smithi007.stdout:1345 read 0x14cac23 thru 0x14cb3c9 (0x7a7 bytes) 2024-09-14T06:59:53.253 INFO:teuthology.orchestra.run.smithi007.stdout:1346 write 0x63de79d thru 0x63eb868 (0xd0cc bytes) 2024-09-14T06:59:53.259 INFO:teuthology.orchestra.run.smithi007.stdout:1347 read 0x470a661 thru 0x471347f (0x8e1f bytes) 2024-09-14T06:59:53.264 INFO:teuthology.orchestra.run.smithi007.stdout:1348 trunc from 0x63eb869 to 0xa445f3e 2024-09-14T06:59:53.270 INFO:teuthology.orchestra.run.smithi007.stdout:1349 trunc from 0xa445f3e to 0x9b3afa1 2024-09-14T06:59:53.273 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:53.273+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:53.275 INFO:teuthology.orchestra.run.smithi007.stdout:1351 writesame 0x70d908d thru 0x70d916b data_size 0x1(0xdf bytes) 2024-09-14T06:59:53.278 INFO:teuthology.orchestra.run.smithi007.stdout:1352 write 0x8cbecc7 thru 0x8cc96cd (0xaa07 bytes) 2024-09-14T06:59:53.292 INFO:teuthology.orchestra.run.smithi007.stdout:1355 write 0xc1d6079 thru 0xc1d9318 (0x32a0 bytes) 2024-09-14T06:59:53.296 INFO:teuthology.orchestra.run.smithi007.stdout:1356 punch from 0xbea48f7 to 0xbeaf455, (0xab5e bytes) 2024-09-14T06:59:53.297 INFO:teuthology.orchestra.run.smithi007.stdout:1358 read 0x37bb510 thru 0x37c5cb6 (0xa7a7 bytes) 2024-09-14T06:59:53.297 INFO:teuthology.orchestra.run.smithi007.stdout:1359 write 0x7b8ffd9 thru 0x7b9b00f (0xb037 bytes) 2024-09-14T06:59:53.304 INFO:teuthology.orchestra.run.smithi007.stdout:1362 read 0x5965346 thru 0x596a814 (0x54cf bytes) 2024-09-14T06:59:53.304 INFO:teuthology.orchestra.run.smithi007.stdout:1363 read 0x83b1108 thru 0x83b5609 (0x4502 bytes) 2024-09-14T06:59:53.304 INFO:teuthology.orchestra.run.smithi007.stdout:1364 punch from 0x9238088 to 0x92444d6, (0xc44e bytes) 2024-09-14T06:59:53.304 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:53.309 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:53.310 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:53.316 INFO:teuthology.orchestra.run.smithi007.stdout:1366 trunc from 0xc1d9319 to 0xf8c6f8 2024-09-14T06:59:53.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:59:53.333 INFO:teuthology.orchestra.run.smithi007.stdout:1368 clone 12 order 18 su 2048 sc 5 2024-09-14T06:59:53.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:59:53.374 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:53.373+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:53.442 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:53.443 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:53.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:59:53.475 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:53.475+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:53.564 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:53.564 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:53.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:53.564 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:53.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.565 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40676 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 761856~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.565 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210659+0000", 2024-09-14T06:59:53.565 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3504872040000002, 2024-09-14T06:59:53.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0085896489999999995, 2024-09-14T06:59:53.565 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40676 2024-09-14T06:59:53.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.567 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210659+0000", 2024-09-14T06:59:53.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210659+0000", 2024-09-14T06:59:53.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210660+0000", 2024-09-14T06:59:53.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-14T06:59:53.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210662+0000", 2024-09-14T06:59:53.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.796e-06 2024-09-14T06:59:53.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210668+0000", 2024-09-14T06:59:53.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2430000000000002e-06 2024-09-14T06:59:53.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210673+0000", 2024-09-14T06:59:53.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8099999999999997e-06 2024-09-14T06:59:53.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219087+0000", 2024-09-14T06:59:53.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084145279999999992 2024-09-14T06:59:53.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219156+0000", 2024-09-14T06:59:53.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9277999999999996e-05 2024-09-14T06:59:53.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219249+0000", 2024-09-14T06:59:53.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2592999999999999e-05 2024-09-14T06:59:53.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.575 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.576 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40678 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 835584~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.576 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:53.576 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3504478970000005, 2024-09-14T06:59:53.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0087422799999999998, 2024-09-14T06:59:53.577 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.577 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.577 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40678 2024-09-14T06:59:53.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.578 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:53.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:53.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:53.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-14T06:59:53.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210701+0000", 2024-09-14T06:59:53.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.81e-06 2024-09-14T06:59:53.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210707+0000", 2024-09-14T06:59:53.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0260000000000004e-06 2024-09-14T06:59:53.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210712+0000", 2024-09-14T06:59:53.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:59:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219281+0000", 2024-09-14T06:59:53.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008569587 2024-09-14T06:59:53.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219351+0000", 2024-09-14T06:59:53.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0055999999999995e-05 2024-09-14T06:59:53.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219441+0000", 2024-09-14T06:59:53.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9608e-05 2024-09-14T06:59:53.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.590 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40681 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 835584~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210737+0000", 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3504097159999997, 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081938849999999997, 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40681 2024-09-14T06:59:53.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210737+0000", 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210737+0000", 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210737+0000", 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210739+0000", 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.685e-06 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210745+0000", 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7790000000000001e-06 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210749+0000", 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4859999999999999e-06 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218765+0000", 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080161420000000004 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218841+0000", 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6114999999999999e-05 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218931+0000", 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9302000000000007e-05 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40705 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 393216~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3503886270000001, 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099565890000000001, 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40705 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210760+0000", 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210768+0000", 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999997e-06 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210773+0000", 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0200000000000002e-06 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219470+0000", 2024-09-14T06:59:53.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0086970339999999993 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219538+0000", 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7550000000000002e-05 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220715+0000", 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001176564 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40706 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 466944~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3503666299999999, 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010139230000000001, 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40706 2024-09-14T06:59:53.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210782+0000", 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210788+0000", 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1480000000000003e-06 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210793+0000", 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220747+0000", 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099533769999999994 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220813+0000", 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5926999999999994e-05 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220919+0000", 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010641099999999999 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.605 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40718 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.605 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:53.605 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3503468500000002, 2024-09-14T06:59:53.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083164229999999999, 2024-09-14T06:59:53.605 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40718 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:59:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210802+0000", 2024-09-14T06:59:53.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.886e-06 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210808+0000", 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0589999999999999e-06 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210815+0000", 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218961+0000", 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081464520000000002 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219029+0000", 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7178e-05 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219116+0000", 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7569000000000001e-05 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40719 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 458752~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210822+0000", 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3503244419999998, 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008481723, 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40719 2024-09-14T06:59:53.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210822+0000", 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210822+0000", 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210823+0000", 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210824+0000", 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.796e-06 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210836+0000", 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1567e-05 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210841+0000", 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3159999999999996e-06 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219153+0000", 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083116370000000002 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219220+0000", 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6958000000000005e-05 2024-09-14T06:59:53.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219304+0000", 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4037999999999997e-05 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40739 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 679936~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210849+0000", 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3502973359999997, 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008625789, 2024-09-14T06:59:53.614 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40739 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210849+0000", 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210849+0000", 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210850+0000", 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210852+0000", 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.153e-06 2024-09-14T06:59:53.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210858+0000", 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4470000000000003e-06 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210863+0000", 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9490000000000004e-06 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219333+0000", 2024-09-14T06:59:53.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084702609999999998 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219398+0000", 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4968999999999994e-05 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219475+0000", 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6625000000000006e-05 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40740 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 753664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210869+0000", 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3502771620000003, 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088293469999999995, 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.619 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40740 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210869+0000", 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210869+0000", 2024-09-14T06:59:53.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210870+0000", 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210886+0000", 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6568000000000001e-05 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210893+0000", 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8759999999999999e-06 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210899+0000", 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.648e-06 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219503+0000", 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0086043809999999995 2024-09-14T06:59:53.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219612+0000", 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109064 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219699+0000", 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6419e-05 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40741 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 827392~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.3502396919999997, 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0089720200000000007, 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40741 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:53.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.75e-07 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210909+0000", 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210916+0000", 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1659999999999999e-06 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210920+0000", 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8640000000000004e-06 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219728+0000", 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088070700000000002 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219794+0000", 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6014999999999997e-05 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219879+0000", 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5370000000000004e-05 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:48248 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 179200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.738998+0000", 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.8221487430000001, 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083283609999999994, 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.628 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 48248 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738998+0000", 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738998+0000", 2024-09-14T06:59:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738998+0000", 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739000+0000", 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0650000000000001e-06 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739006+0000", 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739011+0000", 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.747191+0000", 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081793260000000007 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.747252+0000", 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1674000000000006e-05 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.747326+0000", 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3912000000000007e-05 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53209 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 851968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.042252+0000", 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5188948570000003, 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091733390000000008, 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53209 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042252+0000", 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042252+0000", 2024-09-14T06:59:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042252+0000", 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2799999999999996e-07 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042255+0000", 2024-09-14T06:59:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042264+0000", 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9700000000000005e-06 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042270+0000", 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:59:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051175+0000", 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0089050499999999994 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051284+0000", 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109417 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051425+0000", 2024-09-14T06:59:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000140994 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53384 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.050817+0000", 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5103296479999999, 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008397247, 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53384 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050817+0000", 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050817+0000", 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050817+0000", 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050819+0000", 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0810000000000001e-06 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050826+0000", 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5390000000000004e-06 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050836+0000", 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0529e-05 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.058431+0000", 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075943019999999998 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.058500+0000", 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9558e-05 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.059214+0000", 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071383600000000003 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53424 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.050960+0000", 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.510187009, 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014641082999999999, 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53424 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050960+0000", 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050960+0000", 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050960+0000", 2024-09-14T06:59:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050962+0000", 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1009999999999999e-06 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050969+0000", 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7630000000000003e-06 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050974+0000", 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9389999999999996e-06 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.059248+0000", 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082738120000000002 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.059318+0000", 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0680999999999996e-05 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.065601+0000", 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0062823769999999996 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53425 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 889856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.051001+0000", 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5101459359999998, 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098477459999999992, 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53425 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051001+0000", 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051001+0000", 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051001+0000", 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9800000000000004e-07 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051004+0000", 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.446e-06 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051011+0000", 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7659999999999998e-06 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051017+0000", 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1e-06 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060637+0000", 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096191520000000006 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060709+0000", 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2407e-05 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060848+0000", 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013937699999999999 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.651 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53462 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 884736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.051100+0000", 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5100467599999998, 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099824709999999997, 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53462 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051100+0000", 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051100+0000", 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051100+0000", 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.34e-07 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051103+0000", 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051110+0000", 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0890000000000002e-06 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051115+0000", 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.592e-06 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060877+0000", 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097612459999999995 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060948+0000", 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1293999999999999e-05 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.061082+0000", 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013445199999999999 2024-09-14T06:59:53.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53463 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.051123+0000", 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5100234170000002, 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010210110999999999, 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.656 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53463 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051123+0000", 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051123+0000", 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051124+0000", 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051126+0000", 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2879999999999999e-06 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051133+0000", 2024-09-14T06:59:53.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8940000000000004e-06 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051138+0000", 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4809999999999997e-06 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.061109+0000", 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099709050000000004 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.061179+0000", 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0236999999999999e-05 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.061333+0000", 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015389900000000001 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53763 3.18 3:1a0a685e:::rbd_data.114f8f5a97df.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.209787+0000", 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.351359505, 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010489207, 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53763 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.209787+0000", 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.209787+0000", 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.209789+0000", 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.844e-06 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210712+0000", 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000923524 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210730+0000", 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7793000000000001e-05 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210739+0000", 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6200000000000005e-06 2024-09-14T06:59:53.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210817+0000", 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210952+0000", 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013575600000000001 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:53.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.211554+0000", 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000601181 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.218261+0000", 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076719999999996 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220167+0000", 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001905389 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220229+0000", 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2769999999999997e-05 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220276+0000", 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6869999999999997e-05 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53764 3.18 3:1a0a685e:::rbd_data.114f8f5a97df.0000000000000008:head [truncate 524288] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:53.666 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.210764+0000", 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.350382438, 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097383399999999998, 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53764 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210764+0000", 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210764+0000", 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210765+0000", 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.282e-06 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210769+0000", 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1559999999999999e-06 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210779+0000", 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0874e-05 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210785+0000", 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4280000000000004e-06 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.216013+0000", 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052282400000000003 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.216116+0000", 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010279499999999999 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.216306+0000", 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018991 2024-09-14T06:59:53.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.218694+0000", 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023877770000000002 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220409+0000", 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001715531 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220461+0000", 2024-09-14T06:59:53.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1653000000000002e-05 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220502+0000", 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1693999999999998e-05 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:62460 3.3 3:c626242c:::rbd_data.114ffe9c58f5.000000000000007c:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:52.932764+0000", 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.62838276800000004, 2024-09-14T06:59:53.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008344888, 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 62460 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932764+0000", 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932764+0000", 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932765+0000", 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.305e-06 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932771+0000", 2024-09-14T06:59:53.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0000000000000002e-06 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932783+0000", 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1429e-05 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932788+0000", 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5779999999999998e-06 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:53.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932848+0000", 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.995e-05 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932924+0000", 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5828999999999995e-05 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.934533+0000", 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00160866 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.941041+0000", 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065081820000000004 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.941050+0000", 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5200000000000003e-06 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.941080+0000", 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0209000000000002e-05 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:53.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:53.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.941109+0000", 2024-09-14T06:59:53.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8226e-05 2024-09-14T06:59:53.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:53.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:53.678 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:53.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:59:53.680 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:53.678+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:53.780 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:53.779+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:53.827 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:53.827 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:53.827 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:53.827 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:53.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:59:53.880 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:53.880+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:53.942 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:53.943 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:53.943 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:53.943 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:53.943 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:53.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:59:53.992 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:53.991+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:54.082 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:54.082+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.363692925000002, 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:59:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T06:59:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17618 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.236329+0000", 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8610799870000001, 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016649290000000001, 2024-09-14T06:59:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17618 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236332+0000", 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237113+0000", 2024-09-14T06:59:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078082900000000003 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237131+0000", 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7747000000000001e-05 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237141+0000", 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0137e-05 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237221+0000", 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0482000000000006e-05 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237383+0000", 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016176100000000001 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237737+0000", 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035380999999999998 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:54.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.241638+0000", 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039011739999999999 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252854+0000", 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011216097 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252919+0000", 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5129999999999995e-05 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252979+0000", 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9573e-05 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40284 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:9 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.204456+0000", 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8929529259999995, 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016005640000000002, 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40284 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204457+0000", 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204459+0000", 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204466+0000", 2024-09-14T06:59:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8730000000000001e-06 2024-09-14T06:59:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204472+0000", 2024-09-14T06:59:54.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4779999999999999e-06 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207358+0000", 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028862549999999999 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207430+0000", 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2268000000000007e-05 2024-09-14T06:59:54.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220462+0000", 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013031951999999999 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40646 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8918112919999999, 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017879991000000001, 2024-09-14T06:59:54.117 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40646 2024-09-14T06:59:54.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205605+0000", 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2260000000000002e-06 2024-09-14T06:59:54.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205611+0000", 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3289999999999999e-06 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205622+0000", 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213726+0000", 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008104745 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213792+0000", 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5401999999999995e-05 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223478+0000", 2024-09-14T06:59:54.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096862100000000007 2024-09-14T06:59:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40683 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:54.124 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8917393730000001, 2024-09-14T06:59:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016494009, 2024-09-14T06:59:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40683 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:54.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205672+0000", 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0379999999999998e-06 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205679+0000", 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205684+0000", 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9359999999999998e-06 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221845+0000", 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016161422000000002 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221913+0000", 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7669000000000006e-05 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222164+0000", 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025130600000000002 2024-09-14T06:59:54.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40685 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205690+0000", 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8917189690000002, 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016657634000000001, 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40685 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205691+0000", 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205693+0000", 2024-09-14T06:59:54.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-14T06:59:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205699+0000", 2024-09-14T06:59:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T06:59:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205703+0000", 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5240000000000002e-06 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222190+0000", 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016486752 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222257+0000", 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7095999999999996e-05 2024-09-14T06:59:54.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222348+0000", 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1061000000000001e-05 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40687 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:54.136 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8916997389999999, 2024-09-14T06:59:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016826543999999999, 2024-09-14T06:59:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40687 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205712+0000", 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205718+0000", 2024-09-14T06:59:54.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205723+0000", 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222374+0000", 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016650933999999999 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222445+0000", 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1304999999999996e-05 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222536+0000", 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1104000000000001e-05 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.141 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40720 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205826+0000", 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8915831430000001, 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016893371000000001, 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40720 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205827+0000", 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.58e-07 2024-09-14T06:59:54.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205834+0000", 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2729999999999997e-06 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205841+0000", 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8260000000000004e-06 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205846+0000", 2024-09-14T06:59:54.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1769999999999997e-06 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222562+0000", 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016716178000000002 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222633+0000", 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0461000000000002e-05 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222720+0000", 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6997999999999996e-05 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40721 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8915554409999995, 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044822000000001, 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40721 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205857+0000", 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-06 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205863+0000", 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205868+0000", 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222745+0000", 2024-09-14T06:59:54.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016877293000000002 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222814+0000", 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8806000000000005e-05 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222899+0000", 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4913000000000005e-05 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40742 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8915348749999996, 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017201659000000001, 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40742 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.151 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205892+0000", 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7008e-05 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205899+0000", 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1489999999999997e-06 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205904+0000", 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222923+0000", 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017018229999999999 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222991+0000", 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8545000000000002e-05 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223076+0000", 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4938999999999994e-05 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40743 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:54.155 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205913+0000", 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.8914961430000004, 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343654999999999, 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40743 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205914+0000", 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-07 2024-09-14T06:59:54.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205916+0000", 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0559999999999999e-06 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205922+0000", 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3659999999999997e-06 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205927+0000", 2024-09-14T06:59:54.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8169999999999998e-06 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223101+0000", 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017174452 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223168+0000", 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6756999999999998e-05 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223257+0000", 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8700000000000001e-05 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43363 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6443223769999999, 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015032403, 2024-09-14T06:59:54.160 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43363 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453089+0000", 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9980000000000002e-06 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453096+0000", 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453101+0000", 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8960000000000002e-06 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465082+0000", 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011981083 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465162+0000", 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0588999999999997e-05 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468119+0000", 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002957079 2024-09-14T06:59:54.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43452 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6440546869999997, 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014487193000000001, 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43452 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3299999999999998e-07 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453357+0000", 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8500000000000001e-06 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453363+0000", 2024-09-14T06:59:54.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3239999999999999e-06 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453368+0000", 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2290000000000002e-06 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467645+0000", 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014276812999999999 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467720+0000", 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4492000000000004e-05 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467842+0000", 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012215199999999999 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43453 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 78848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:54.169 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6440344759999999, 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015012878, 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43453 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:54.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453377+0000", 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1050000000000002e-06 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453384+0000", 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453389+0000", 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-06 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467869+0000", 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014479845999999999 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467944+0000", 2024-09-14T06:59:54.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5680999999999996e-05 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468388+0000", 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000443473 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43576 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 99328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:54.174 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6416016110000005, 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014560714000000001, 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43576 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:54.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455809+0000", 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.091e-06 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455812+0000", 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199999999999999e-06 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455815+0000", 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.537e-06 2024-09-14T06:59:54.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470167+0000", 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014352746 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470245+0000", 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7730000000000003e-05 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470368+0000", 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012338 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43577 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 102400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6415915300000004, 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014789722999999999, 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43577 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455820+0000", 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4270000000000001e-06 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455823+0000", 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0139999999999999e-06 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455825+0000", 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4980000000000001e-06 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470402+0000", 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014576858999999999 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470485+0000", 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2731000000000005e-05 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470608+0000", 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012297299999999999 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43578 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6415708740000001, 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015008412, 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.184 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43578 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4400000000000001e-07 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455841+0000", 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2749999999999998e-06 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455848+0000", 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9349999999999997e-06 2024-09-14T06:59:54.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455854+0000", 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-06 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470639+0000", 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014785382999999999 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470716+0000", 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7155000000000002e-05 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470847+0000", 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013058000000000001 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43579 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:54.188 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.641547514, 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01522916, 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43579 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455864+0000", 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455870+0000", 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455875+0000", 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9509999999999997e-06 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470879+0000", 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015003729 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470960+0000", 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1228000000000002e-05 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471091+0000", 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131018 2024-09-14T06:59:54.192 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43580 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 110592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455881+0000", 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6415279759999999, 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015447047, 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43580 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455882+0000", 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455884+0000", 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2010000000000002e-06 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455891+0000", 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5690000000000001e-06 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455896+0000", 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3639999999999998e-06 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471123+0000", 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015226805 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471200+0000", 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7405999999999997e-05 2024-09-14T06:59:54.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471328+0000", 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012835099999999999 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43581 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455903+0000", 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6415060749999997, 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015668949000000001, 2024-09-14T06:59:54.197 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43581 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T06:59:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455904+0000", 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455910+0000", 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6619999999999999e-06 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455917+0000", 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5769999999999999e-06 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455923+0000", 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6779999999999997e-06 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471360+0000", 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015436854999999999 2024-09-14T06:59:54.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471443+0000", 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3815999999999998e-05 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471572+0000", 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012892900000000001 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.204 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:54.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:59:54.206 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:54.204+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:54.306 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:54.305+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:54.313 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:54.313 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:54.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:59:54.335 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T06:59:54.407 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:54.406+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:54.427 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone11 from 0xf8c6f8 (overlap 0xf8c6f8) to 0x6382c9 2024-09-14T06:59:54.450 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:54.450 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:54.450 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:54.450 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:54.450 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:54.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:59:54.508 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:54.507+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:54.589 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:54.589 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:15473 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.124194+0000", 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.462980637999999, 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052649906000000003, 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15473 2024-09-14T06:59:54.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124194+0000", 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124194+0000", 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124195+0000", 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2700000000000001e-07 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124196+0000", 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1650000000000001e-06 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124199+0000", 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.269e-06 2024-09-14T06:59:54.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124202+0000", 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6000000000000001e-06 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176600+0000", 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052398307999999998 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176668+0000", 2024-09-14T06:59:54.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7859000000000002e-05 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176844+0000", 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017647800000000001 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:15474 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.124205+0000", 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.462970116999999, 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053002028999999999, 2024-09-14T06:59:54.594 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15474 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124205+0000", 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124205+0000", 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130009+0000", 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058039299999999997 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130013+0000", 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7129999999999999e-06 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130021+0000", 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7749999999999998e-06 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130026+0000", 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.938e-06 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176891+0000", 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046864380999999997 2024-09-14T06:59:54.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176972+0000", 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1581000000000004e-05 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.177207+0000", 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000234711 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18567 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 12926976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319102+0000", 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.268073144000001, 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052720059999999999, 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18567 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319102+0000", 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319102+0000", 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319102+0000", 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319104+0000", 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.356e-06 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319107+0000", 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6040000000000001e-06 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319110+0000", 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8729999999999999e-06 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371714+0000", 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052603556000000003 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371770+0000", 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6097999999999997e-05 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371822+0000", 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2207999999999999e-05 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18568 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13041664~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319114+0000", 2024-09-14T06:59:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.268061472999999, 2024-09-14T06:59:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052952595999999998, 2024-09-14T06:59:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18568 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319114+0000", 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319114+0000", 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319114+0000", 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319115+0000", 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319118+0000", 2024-09-14T06:59:54.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0599999999999999e-06 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319121+0000", 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7319999999999998e-06 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371870+0000", 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052749273999999999 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371959+0000", 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9061000000000007e-05 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372066+0000", 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107299 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18569 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13156352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319124+0000", 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.268051014999999, 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053177681999999997, 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18569 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319124+0000", 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319124+0000", 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319124+0000", 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:59:54.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319125+0000", 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319129+0000", 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9910000000000004e-06 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319132+0000", 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8119999999999999e-06 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372102+0000", 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052970060999999999 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372192+0000", 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9778999999999995e-05 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372302+0000", 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001099 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18570 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13271040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319135+0000", 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.268039731, 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053394978000000003, 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18570 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319135+0000", 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319135+0000", 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319136+0000", 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2399999999999999e-07 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319137+0000", 2024-09-14T06:59:54.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.206e-06 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319140+0000", 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1159999999999999e-06 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319143+0000", 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6800000000000002e-06 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372337+0000", 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194033000000002 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372424+0000", 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7880000000000006e-05 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372530+0000", 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105839 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18571 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13385728~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319148+0000", 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.268026581999999, 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053601401999999999, 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18571 2024-09-14T06:59:54.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319148+0000", 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319148+0000", 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319149+0000", 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319150+0000", 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2160000000000001e-06 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319155+0000", 2024-09-14T06:59:54.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0520000000000001e-06 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319159+0000", 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3329999999999998e-06 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372564+0000", 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053404576000000002 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372652+0000", 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7719000000000005e-05 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372750+0000", 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8256000000000003e-05 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18572 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13500416~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319165+0000", 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.268010146, 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053777864000000002, 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18572 2024-09-14T06:59:54.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319165+0000", 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319165+0000", 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319165+0000", 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319166+0000", 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.172e-06 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319170+0000", 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3589999999999999e-06 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319172+0000", 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6740000000000001e-06 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372792+0000", 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053620062000000003 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372883+0000", 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0847000000000006e-05 2024-09-14T06:59:54.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372943+0000", 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9469000000000003e-05 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18573 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13615104~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319175+0000", 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.267999557, 2024-09-14T06:59:54.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054002965, 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18573 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319175+0000", 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319175+0000", 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319176+0000", 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319177+0000", 2024-09-14T06:59:54.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319182+0000", 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0919999999999997e-06 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319186+0000", 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372988+0000", 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053801579000000002 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373077+0000", 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.954e-05 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373178+0000", 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101271 2024-09-14T06:59:54.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18574 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13729792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319192+0000", 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.267982846000001, 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054130926000000003, 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18574 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319192+0000", 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319192+0000", 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319192+0000", 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319194+0000", 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319199+0000", 2024-09-14T06:59:54.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2909999999999998e-06 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319203+0000", 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5469999999999998e-06 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373210+0000", 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054006709999999999 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373270+0000", 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0229999999999998e-05 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373323+0000", 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2675e-05 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18575 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13844480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319210+0000", 2024-09-14T06:59:54.638 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.267965111000001, 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054350181999999997, 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18575 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319210+0000", 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319210+0000", 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319210+0000", 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7500000000000001e-07 2024-09-14T06:59:54.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319212+0000", 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0640000000000001e-06 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319219+0000", 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2489999999999998e-06 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319224+0000", 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0170000000000004e-06 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373364+0000", 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054140677999999998 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373452+0000", 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7497000000000006e-05 2024-09-14T06:59:54.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373560+0000", 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010830199999999999 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18576 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13959168~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319231+0000", 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.267944361, 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054486288000000001, 2024-09-14T06:59:54.643 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18576 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319231+0000", 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319231+0000", 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319231+0000", 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319233+0000", 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0439999999999998e-06 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319237+0000", 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7560000000000001e-06 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319240+0000", 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373591+0000", 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054351337999999999 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373647+0000", 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5300000000000002e-05 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373717+0000", 2024-09-14T06:59:54.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0276000000000003e-05 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18577 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 14073856~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319265+0000", 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.267909795, 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054723081999999999, 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18577 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319265+0000", 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319265+0000", 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319265+0000", 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319267+0000", 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T06:59:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319271+0000", 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3200000000000001e-06 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319274+0000", 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1499999999999999e-06 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373772+0000", 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054497800999999998 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373883+0000", 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110654 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373988+0000", 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105404 2024-09-14T06:59:54.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18578 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 14188544~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319279+0000", 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.267896288999999, 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054840223, 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18578 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319279+0000", 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319279+0000", 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324912+0000", 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056328250000000002 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324916+0000", 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0330000000000002e-06 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324926+0000", 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0379e-05 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324931+0000", 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374008+0000", 2024-09-14T06:59:54.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.049076501000000002 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374065+0000", 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6753999999999999e-05 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374119+0000", 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4441999999999998e-05 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18645 3.16 3:6af7eeae:::rbd_data.114f54fbf0ed.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.324991+0000", 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.262184442000001, 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054731618000000003, 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18645 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324991+0000", 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324991+0000", 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324991+0000", 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9700000000000003e-07 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326138+0000", 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011472909999999999 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326147+0000", 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8089999999999999e-06 2024-09-14T06:59:54.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326152+0000", 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374799+0000", 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.048647060999999998 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374946+0000", 2024-09-14T06:59:54.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014760700000000001 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.375283+0000", 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033679499999999998 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379203+0000", 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039194779999999997 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379627+0000", 2024-09-14T06:59:54.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042489399999999999 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379683+0000", 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5037999999999997e-05 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379722+0000", 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9682999999999999e-05 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18809 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 10985472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326645+0000", 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.260530109999999, 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052693862000000001, 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18809 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326645+0000", 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326645+0000", 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326645+0000", 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326651+0000", 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2129999999999996e-06 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326656+0000", 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8860000000000003e-06 2024-09-14T06:59:54.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326659+0000", 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379202+0000", 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052542356999999998 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379280+0000", 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8115999999999995e-05 2024-09-14T06:59:54.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379339+0000", 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9104999999999997e-05 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18810 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11100160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326663+0000", 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.260511922999999, 2024-09-14T06:59:54.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052824810999999999, 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18810 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326663+0000", 2024-09-14T06:59:54.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326663+0000", 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326663+0000", 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326664+0000", 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1179999999999999e-06 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326669+0000", 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7600000000000002e-06 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326674+0000", 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3499999999999999e-06 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379369+0000", 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052695288999999999 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379427+0000", 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8074e-05 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379488+0000", 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0977999999999998e-05 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.672 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18811 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11214848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326680+0000", 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.260495206, 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053046902, 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18811 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326680+0000", 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326680+0000", 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326680+0000", 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5400000000000002e-07 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326682+0000", 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9429999999999999e-06 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326688+0000", 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6269999999999997e-06 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326691+0000", 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8990000000000001e-06 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379514+0000", 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052823399 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379559+0000", 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4613000000000001e-05 2024-09-14T06:59:54.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379727+0000", 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016806699999999999 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18812 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11329536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326694+0000", 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.260480557999999, 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053113457000000003, 2024-09-14T06:59:54.677 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18812 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326694+0000", 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326694+0000", 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326695+0000", 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2600000000000001e-07 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326696+0000", 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1999999999999999e-06 2024-09-14T06:59:54.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326699+0000", 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.101e-06 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326702+0000", 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7630000000000001e-06 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379684+0000", 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052982143000000002 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379756+0000", 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2056000000000003e-05 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379808+0000", 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1968000000000001e-05 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18813 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11444224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326705+0000", 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.260470206000001, 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053264406, 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18813 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.682 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326705+0000", 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326705+0000", 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331872+0000", 2024-09-14T06:59:54.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051673459999999997 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331877+0000", 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5460000000000002e-06 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331887+0000", 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0078000000000001e-05 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331892+0000", 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379878+0000", 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047986046999999997 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379919+0000", 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:54.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:54.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379969+0000", 2024-09-14T06:59:54.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0185999999999997e-05 2024-09-14T06:59:54.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:54.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:54.686 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:54.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:59:54.689 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-14T06:59:54.689 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:54.689 INFO:teuthology.orchestra.run.smithi007.stdout:1370 read 0x33268b thru 0x334ed3 (0x2849 bytes) 2024-09-14T06:59:54.689 INFO:teuthology.orchestra.run.smithi007.stdout:1371 trunc from 0xf8c6f8 to 0xe263fd4 2024-09-14T06:59:54.689 INFO:teuthology.orchestra.run.smithi007.stdout:1372 writesame 0x83f5e21 thru 0x8405722 data_size 0x1(0xf902 bytes) 2024-09-14T06:59:54.689 INFO:teuthology.orchestra.run.smithi007.stdout:1373 punch from 0x628d61f to 0x629d449, (0xfe2a bytes) 2024-09-14T06:59:54.689 INFO:teuthology.orchestra.run.smithi007.stdout:1375 write 0xaf9ecdb thru 0xafadaff (0xee25 bytes) 2024-09-14T06:59:54.689 INFO:teuthology.orchestra.run.smithi007.stdout:1377 write 0x54e9e81 thru 0x54f8144 (0xe2c4 bytes) 2024-09-14T06:59:54.689 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:54.686+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:54.690 INFO:teuthology.orchestra.run.smithi007.stdout:1379 write 0xc891a7e thru 0xc893b1b (0x209e bytes) 2024-09-14T06:59:54.695 INFO:teuthology.orchestra.run.smithi007.stdout:1380 writesame 0xd6e7eee thru 0xd6e88cd data_size 0x10(0x9e0 bytes) 2024-09-14T06:59:54.699 INFO:teuthology.orchestra.run.smithi007.stdout:1381 write 0xd38da03 thru 0xd395b1d (0x811b bytes) 2024-09-14T06:59:54.705 INFO:teuthology.orchestra.run.smithi007.stdout:1383 write 0x15ebe76 thru 0x15f9b37 (0xdcc2 bytes) 2024-09-14T06:59:54.711 INFO:teuthology.orchestra.run.smithi007.stdout:1384 writesame 0xaf45c8a thru 0xaf4b343 data_size 0x11(0x56ba bytes) 2024-09-14T06:59:54.714 INFO:teuthology.orchestra.run.smithi007.stdout:1386 read 0xbd91f69 thru 0xbda1366 (0xf3fe bytes) 2024-09-14T06:59:54.714 INFO:teuthology.orchestra.run.smithi007.stdout:1387 read 0xc2c6bbb thru 0xc2cd7d1 (0x6c17 bytes) 2024-09-14T06:59:54.714 INFO:teuthology.orchestra.run.smithi007.stdout:1388 punch from 0x7d553cb to 0x7d6507f, (0xfcb4 bytes) 2024-09-14T06:59:54.714 INFO:teuthology.orchestra.run.smithi007.stdout:1391 write 0x5c377af thru 0x5c42b8a (0xb3dc bytes) 2024-09-14T06:59:54.723 INFO:teuthology.orchestra.run.smithi007.stdout:1392 writesame 0xbbdcb7e thru 0xbbe8998 data_size 0x1(0xbe1b bytes) 2024-09-14T06:59:54.736 INFO:teuthology.orchestra.run.smithi007.stdout:1393 writesame 0xd35583c thru 0xd35a799 data_size 0x1(0x4f5e bytes) 2024-09-14T06:59:54.741 INFO:teuthology.orchestra.run.smithi007.stdout:1394 read 0x5db2d0c thru 0x5db361e (0x913 bytes) 2024-09-14T06:59:54.741 INFO:teuthology.orchestra.run.smithi007.stdout:1395 read 0xbdc5dfa thru 0xbdca59d (0x47a4 bytes) 2024-09-14T06:59:54.741 INFO:teuthology.orchestra.run.smithi007.stdout:1396 writesame 0x7bfb37e thru 0x7c02331 data_size 0x6(0x6fb4 bytes) 2024-09-14T06:59:54.752 INFO:teuthology.orchestra.run.smithi007.stdout:1398 read 0xe1983e9 thru 0xe19b89b (0x34b3 bytes) 2024-09-14T06:59:54.752 INFO:teuthology.orchestra.run.smithi007.stdout:1400 write 0xdf151c4 thru 0xdf21395 (0xc1d2 bytes) 2024-09-14T06:59:54.758 INFO:teuthology.orchestra.run.smithi007.stdout:1401 write 0xab26dcf thru 0xab2aecc (0x40fe bytes) 2024-09-14T06:59:54.764 INFO:teuthology.orchestra.run.smithi007.stdout:1402 read 0x2e460a9 thru 0x2e54240 (0xe198 bytes) 2024-09-14T06:59:54.764 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:54.764 INFO:teuthology.orchestra.run.smithi007.stdout:1404 punch from 0x715636f to 0x715d1f3, (0x6e84 bytes) 2024-09-14T06:59:54.764 INFO:teuthology.orchestra.run.smithi007.stdout:1405 read 0xe21f200 thru 0xe229222 (0xa023 bytes) 2024-09-14T06:59:54.764 INFO:teuthology.orchestra.run.smithi007.stdout:1408 writesame 0x96415e4 thru 0x964a749 data_size 0x2(0x9166 bytes) 2024-09-14T06:59:54.773 INFO:teuthology.orchestra.run.smithi007.stdout:1409 write 0x5350e7e thru 0x5354968 (0x3aeb bytes) 2024-09-14T06:59:54.782 INFO:teuthology.orchestra.run.smithi007.stdout:1412 read 0x9c5c44f thru 0x9c68f76 (0xcb28 bytes) 2024-09-14T06:59:54.782 INFO:teuthology.orchestra.run.smithi007.stdout:1415 writesame 0xdd20c54 thru 0xdd25726 data_size 0x5(0x4ad3 bytes) 2024-09-14T06:59:54.788 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:54.788+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:54.792 INFO:teuthology.orchestra.run.smithi007.stdout:1416 read 0xcc167a thru 0xcc6696 (0x501d bytes) 2024-09-14T06:59:54.794 INFO:teuthology.orchestra.run.smithi007.stdout:1417 clone 13 order 23 su 32768 sc 3 2024-09-14T06:59:54.817 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:54.818 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:54.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:59:54.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:59:54.909 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:54.909+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:54.976 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:54.976 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:54.976 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:54.976 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:54.976 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:54.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:59:54.989 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:54.989+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:54.991 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:54.992 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:55.009 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:55.026 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:55.027 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:55.043 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:55.043 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:55.060 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:55.060 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:55.077 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:55.077 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:55.090 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:55.090+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:55.093 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:55.093 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:55.100 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:55.101 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:55.109 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:59:55.110 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:55.110 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:55.192 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:55.191+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:55.212 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:55.212 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:55.221 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:59:55.305 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:55.304+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:55.348 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:59:55.349 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:59:55.349 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T06:59:55.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:59:55.367 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:55.367 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:55.376 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:59:55.418 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:55.417+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:55.443 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone12 from 0xe263fd4 (overlap 0xf8c6f8) to 0xdfe645 2024-09-14T06:59:55.495 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:55.494+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:55.539 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:55.539 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:55.548 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:59:55.634 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:55.633+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:55.643 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-14T06:59:55.697 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:55.696+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:55.712 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:55.712 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:55.721 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:59:55.722 INFO:teuthology.orchestra.run.smithi007.stdout:1418 write 0xd4d4413 thru 0xd4dddb5 (0x99a3 bytes) 2024-09-14T06:59:55.722 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:55.723 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:55.723 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 46, 2024-09-14T06:59:55.723 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6", 2024-09-14T06:59:55.723 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-14T06:58:59.479055+0000", 2024-09-14T06:59:55.723 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-14T06:59:55.346155+0000", 2024-09-14T06:59:55.723 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-14T06:59:11.718934+0000", 2024-09-14T06:59:55.723 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-14T06:59:50.317515+0000", 2024-09-14T06:59:55.723 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5806080, 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 6, 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:59:55.724 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-14T06:59:55.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:11.765419+0000", 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:59:55.726 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T06:59:55.727 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "18", 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-14T06:59:55.728 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T06:59:55.729 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T06:59:55.730 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T06:59:55.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 6.9000000953674316, 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 6.9000000953674316, 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:59:55.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:18.343475+0000", 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:59:55.733 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T06:59:55.734 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "22", 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-14T06:59:55.735 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 22, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:59:55.736 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:59:55.737 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 4.380000114440918, 2024-09-14T06:59:55.738 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 4.380000114440918, 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:27.257805+0000", 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T06:59:55.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-14T06:59:55.740 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "46", 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T06:59:55.741 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 15, 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 46, 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T06:59:55.742 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T06:59:55.743 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.744 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.75, 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.75, 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.75, 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.75, 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:55.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c036038d-cffb-4584-8851-69604a31ee40", 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:55.746 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.747 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.748 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:55.749 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T06:59:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/2081065366", 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/2081065366", 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/2081065366", 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/2081065366", 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T06:59:55.751 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "1c80080f-932a-4957-b64b-f30c4bdb9340", 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.756 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6806/1688050909", 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6810/1688050909", 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/1688050909", 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/1688050909", 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f3544d11-8a0f-498c-a03c-5d34134c9773", 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:55.757 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-14T06:59:55.758 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-14T06:59:55.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/1872259061", 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/1872259061", 2024-09-14T06:59:55.762 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/1872259061", 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/1872259061", 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d2261455-c91b-414b-925f-388f0e094dbe", 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:55.763 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 44, 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/1193301030", 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/1193301030", 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6811/1193301030", 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6807/1193301030", 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4088d23b-0d06-4415-9eba-6fb64355746c", 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:55.769 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6810", 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6811", 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.770 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6812", 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6813", 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6816", 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6817", 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:55.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6814", 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6815", 2024-09-14T06:59:55.773 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6811/2283562461", 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6813/2283562461", 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6817/2283562461", 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6815/2283562461", 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.774 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "5e27cb53-ecce-4061-bc5a-be599bf18cc4", 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:55.775 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6818", 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6819", 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:55.776 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6820", 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6821", 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.777 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6824", 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6825", 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6822", 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6823", 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6819/2212769626", 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6821/2212769626", 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6825/2212769626", 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6823/2212769626", 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ceb9a1f1-f54c-48d5-91d8-2287cac96ceb", 2024-09-14T06:59:55.780 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.781 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6826", 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6827", 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6828", 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6829", 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6832", 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6833", 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6830", 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6831", 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6827/3197231389", 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6829/3197231389", 2024-09-14T06:59:55.785 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6833/3197231389", 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6831/3197231389", 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85c59cd6-6e2b-4cc0-852d-d1f075a8826d", 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T06:59:55.786 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6800", 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.788 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6801", 2024-09-14T06:59:55.788 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:55.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.788 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.788 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6802", 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6803", 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6806", 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6807", 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6804", 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6805", 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6801/2162534312", 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6803/2162534312", 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6807/2162534312", 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6805/2162534312", 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T06:59:55.792 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.064941+0000", 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:55.793 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.780598+0000", 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.127700+0000", 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T06:59:55.794 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.730995+0000", 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:55.795 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.864423+0000", 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.329361+0000", 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T06:59:55.796 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:10.107581+0000", 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T06:59:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.300043+0000", 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T06:59:55.798 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:55.799 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:55.800 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:55.803 INFO:tasks.thrashosds.thrasher:j is {'epoch': 46, 'fsid': '32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6', 'created': '2024-09-14T06:58:59.479055+0000', 'modified': '2024-09-14T06:59:55.346155+0000', 'last_up_change': '2024-09-14T06:59:11.718934+0000', 'last_in_change': '2024-09-14T06:59:50.317515+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:59:11.765419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:59:18.343475+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.380000114440918, 'score_stable': 4.380000114440918, 'optimal_score': 1, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:59:27.257805+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 46, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c036038d-cffb-4584-8851-69604a31ee40', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 2081065366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 2081065366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 2081065366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 2081065366}]}, 'public_addr': '172.21.15.7:6817/2081065366', 'cluster_addr': '172.21.15.7:6819/2081065366', 'heartbeat_back_addr': '172.21.15.7:6823/2081065366', 'heartbeat_front_addr': '172.21.15.7:6821/2081065366', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1c80080f-932a-4957-b64b-f30c4bdb9340', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6806', 'nonce': 1688050909}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6810', 'nonce': 1688050909}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 1688050909}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 1688050909}]}, 'public_addr': '172.21.15.7:6806/1688050909', 'cluster_addr': '172.21.15.7:6810/1688050909', 'heartbeat_back_addr': '172.21.15.7:6815/1688050909', 'heartbeat_front_addr': '172.21.15.7:6813/1688050909', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f3544d11-8a0f-498c-a03c-5d34134c9773', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 1872259061}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 1872259061}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 1872259061}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 1872259061}]}, 'public_addr': '172.21.15.7:6825/1872259061', 'cluster_addr': '172.21.15.7:6827/1872259061', 'heartbeat_back_addr': '172.21.15.7:6831/1872259061', 'heartbeat_front_addr': '172.21.15.7:6829/1872259061', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd2261455-c91b-414b-925f-388f0e094dbe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 1193301030}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 1193301030}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6809', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 1193301030}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6805', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 1193301030}]}, 'public_addr': '172.21.15.7:6801/1193301030', 'cluster_addr': '172.21.15.7:6803/1193301030', 'heartbeat_back_addr': '172.21.15.7:6811/1193301030', 'heartbeat_front_addr': '172.21.15.7:6807/1193301030', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4088d23b-0d06-4415-9eba-6fb64355746c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6810', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6811', 'nonce': 2283562461}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6812', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6813', 'nonce': 2283562461}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6816', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6817', 'nonce': 2283562461}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6814', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6815', 'nonce': 2283562461}]}, 'public_addr': '172.21.15.189:6811/2283562461', 'cluster_addr': '172.21.15.189:6813/2283562461', 'heartbeat_back_addr': '172.21.15.189:6817/2283562461', 'heartbeat_front_addr': '172.21.15.189:6815/2283562461', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5e27cb53-ecce-4061-bc5a-be599bf18cc4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6818', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6819', 'nonce': 2212769626}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6820', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6821', 'nonce': 2212769626}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6824', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6825', 'nonce': 2212769626}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6822', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6823', 'nonce': 2212769626}]}, 'public_addr': '172.21.15.189:6819/2212769626', 'cluster_addr': '172.21.15.189:6821/2212769626', 'heartbeat_back_addr': '172.21.15.189:6825/2212769626', 'heartbeat_front_addr': '172.21.15.189:6823/2212769626', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceb9a1f1-f54c-48d5-91d8-2287cac96ceb', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6826', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6827', 'nonce': 3197231389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6828', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6829', 'nonce': 3197231389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 3197231389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6830', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6831', 'nonce': 3197231389}]}, 'public_addr': '172.21.15.189:6827/3197231389', 'cluster_addr': '172.21.15.189:6829/3197231389', 'heartbeat_back_addr': '172.21.15.189:6833/3197231389', 'heartbeat_front_addr': '172.21.15.189:6831/3197231389', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '85c59cd6-6e2b-4cc0-852d-d1f075a8826d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6800', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6801', 'nonce': 2162534312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6802', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6803', 'nonce': 2162534312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6806', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6807', 'nonce': 2162534312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6804', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6805', 'nonce': 2162534312}]}, 'public_addr': '172.21.15.189:6801/2162534312', 'cluster_addr': '172.21.15.189:6803/2162534312', 'heartbeat_back_addr': '172.21.15.189:6807/2162534312', 'heartbeat_front_addr': '172.21.15.189:6805/2162534312', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.064941+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.780598+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.127700+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.730995+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.864423+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.329361+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T06:59:10.107581+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.300043+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.18', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:59:55.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:59:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:1420 read 0x2d7dd67 thru 0x2d88327 (0xa5c1 bytes) 2024-09-14T06:59:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:1422 read 0xbaf85ea thru 0xbb003c5 (0x7ddc bytes) 2024-09-14T06:59:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:1423 write 0xeb52734 thru 0xeb58146 (0x5a13 bytes) 2024-09-14T06:59:55.804 INFO:teuthology.orchestra.run.smithi007.stdout:1424 trunc from 0xeb58147 to 0xb3564ba 2024-09-14T06:59:55.808 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:55.801+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:1425 read 0x72d509f thru 0x72dbf85 (0x6ee7 bytes) 2024-09-14T06:59:55.829 INFO:teuthology.orchestra.run.smithi007.stdout:1428 read 0x26c89f thru 0x27632a (0x9a8c bytes) 2024-09-14T06:59:55.833 INFO:teuthology.orchestra.run.smithi007.stdout:1430 read 0x52c0897 thru 0x52d04f1 (0xfc5b bytes) 2024-09-14T06:59:55.834 INFO:teuthology.orchestra.run.smithi007.stdout:1433 punch from 0x1f586ad to 0x1f63f00, (0xb853 bytes) 2024-09-14T06:59:55.838 INFO:teuthology.orchestra.run.smithi007.stdout:1434 trunc from 0xb3564ba to 0xe7634bb 2024-09-14T06:59:55.843 INFO:teuthology.orchestra.run.smithi007.stdout:1435 read 0x8f0fa16 thru 0x8f1f965 (0xff50 bytes) 2024-09-14T06:59:55.855 INFO:teuthology.orchestra.run.smithi007.stdout:1436 trunc from 0xe7634bb to 0x3fae18d 2024-09-14T06:59:55.880 INFO:teuthology.orchestra.run.smithi007.stdout:1437 trunc from 0x3fae18d to 0x1b77c4d 2024-09-14T06:59:55.902 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:55.902+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:55.908 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:55.908 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:55.917 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:59:55.922 INFO:teuthology.orchestra.run.smithi007.stdout:1438 write 0xe026d15 thru 0xe02d390 (0x667c bytes) 2024-09-14T06:59:55.926 INFO:teuthology.orchestra.run.smithi007.stdout:1440 write 0x58d00d1 thru 0x58d7d3e (0x7c6e bytes) 2024-09-14T06:59:55.929 INFO:teuthology.orchestra.run.smithi007.stdout:1441 read 0xddd986e thru 0xdde2840 (0x8fd3 bytes) 2024-09-14T06:59:55.929 INFO:teuthology.orchestra.run.smithi007.stdout:1442 writesame 0xb05d6d0 thru 0xb069498 data_size 0x7b(0xbdc9 bytes) 2024-09-14T06:59:55.934 INFO:teuthology.orchestra.run.smithi007.stdout:1444 read 0x43e2454 thru 0x43e92cc (0x6e79 bytes) 2024-09-14T06:59:55.934 INFO:teuthology.orchestra.run.smithi007.stdout:1445 writesame 0xb14c125 thru 0xb14ef95 data_size 0x3(0x2e71 bytes) 2024-09-14T06:59:55.936 INFO:teuthology.orchestra.run.smithi007.stdout:1446 punch from 0xc87212 to 0xc90116, (0x8f04 bytes) 2024-09-14T06:59:55.942 INFO:teuthology.orchestra.run.smithi007.stdout:1448 write 0xe12ff67 thru 0xe13f4eb (0xf585 bytes) 2024-09-14T06:59:55.945 INFO:teuthology.orchestra.run.smithi007.stdout:1449 clone 14 order 18 su 16384 sc 14 2024-09-14T06:59:56.003 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:56.003+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:56.061 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:56.061 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:56.069 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:59:56.113 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:56.112+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:56.132 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T06:59:56.133 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T06:59:56.144 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-14T06:59:55.756350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":438614667,"num_objects":498,"num_object_clones":107,"num_object_copies":996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":498,"num_whiteouts":208,"num_read":69172,"num_read_kb":187782,"num_write":3420,"num_write_kb":114796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":3048404,"num_keys_recovered":23,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3039,"ondisk_log_size":3039,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":246884,"kb_used_data":16024,"kb_used_omap":10,"kb_used_meta":230645,"kb_avail":660355996,"statfs":{"total":676457349120,"available":676204539904,"internally_reserved":0,"allocated":16408576,"data_stored":123233356,"data_compressed":3821940,"data_compressed_allocated":10153984,"data_compressed_original":120217436,"omap_allocated":11157,"internal_metadata":236180587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47173515,"num_objects":162,"num_object_clones":39,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":105,"num_read":25416,"num_read_kb":51780,"num_write":305,"num_write_kb":19841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":103,"num_bytes_recovered":3048404,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000672"},"pg_stats":[{"pgid":"3.1f","version":"46'144","reported_seq":4512,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T06:59:55.511513+0000","last_change":"2024-09-14T06:59:30.808998+0000","last_active":"2024-09-14T06:59:55.511513+0000","last_peered":"2024-09-14T06:59:55.511513+0000","last_clean":"2024-09-14T06:59:55.511513+0000","last_became_active":"2024-09-14T06:59:28.801961+0000","last_became_peered":"2024-09-14T06:59:28.801961+0000","last_unstale":"2024-09-14T06:59:55.511513+0000","last_undegraded":"2024-09-14T06:59:55.511513+0000","last_fullsized":"2024-09-14T06:59:55.511513+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:30:29.304169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043615199999999998,"stat_sum":{"num_bytes":34669686,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":11,"num_read":4889,"num_read_kb":18413,"num_write":137,"num_write_kb":13405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"46'72","reported_seq":173,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T06:59:55.474050+0000","last_change":"2024-09-14T06:59:30.808971+0000","last_active":"2024-09-14T06:59:55.474050+0000","last_peered":"2024-09-14T06:59:55.474050+0000","last_clean":"2024-09-14T06:59:55.474050+0000","last_became_active":"2024-09-14T06:59:28.801728+0000","last_became_peered":"2024-09-14T06:59:28.801728+0000","last_unstale":"2024-09-14T06:59:55.474050+0000","last_undegraded":"2024-09-14T06:59:55.474050+0000","last_fullsized":"2024-09-14T06:59:55.474050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:52.619009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000412824,"stat_sum":{"num_bytes":2065214,"num_objects":20,"num_object_clones":5,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":12,"num_read":58,"num_read_kb":354,"num_write":62,"num_write_kb":504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"40'46","reported_seq":4230,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.495307+0000","last_change":"2024-09-14T06:59:30.803414+0000","last_active":"2024-09-14T06:59:52.495307+0000","last_peered":"2024-09-14T06:59:52.495307+0000","last_clean":"2024-09-14T06:59:52.495307+0000","last_became_active":"2024-09-14T06:59:28.799714+0000","last_became_peered":"2024-09-14T06:59:28.799714+0000","last_unstale":"2024-09-14T06:59:52.495307+0000","last_undegraded":"2024-09-14T06:59:52.495307+0000","last_fullsized":"2024-09-14T06:59:52.495307+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:32.527401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001689915,"stat_sum":{"num_bytes":5315403,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":4154,"num_read_kb":9726,"num_write":43,"num_write_kb":4041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"43'44","reported_seq":217,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.483119+0000","last_change":"2024-09-14T06:59:30.806817+0000","last_active":"2024-09-14T06:59:52.483119+0000","last_peered":"2024-09-14T06:59:52.483119+0000","last_clean":"2024-09-14T06:59:52.483119+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T06:59:52.483119+0000","last_undegraded":"2024-09-14T06:59:52.483119+0000","last_fullsized":"2024-09-14T06:59:52.483119+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:49:00.730595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042050300000000002,"stat_sum":{"num_bytes":2054920,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":146,"num_read_kb":567,"num_write":45,"num_write_kb":1157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"40'128","reported_seq":297,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-14T06:59:51.335167+0000","last_change":"2024-09-14T06:59:30.801680+0000","last_active":"2024-09-14T06:59:51.335167+0000","last_peered":"2024-09-14T06:59:51.335167+0000","last_clean":"2024-09-14T06:59:51.335167+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T06:59:51.335167+0000","last_undegraded":"2024-09-14T06:59:51.335167+0000","last_fullsized":"2024-09-14T06:59:51.335167+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:36:20.152972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060885900000000001,"stat_sum":{"num_bytes":15043508,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":10,"num_read":408,"num_read_kb":825,"num_write":150,"num_write_kb":3317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"43'90","reported_seq":3454,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.487664+0000","last_change":"2024-09-14T06:59:30.807597+0000","last_active":"2024-09-14T06:59:52.487664+0000","last_peered":"2024-09-14T06:59:52.487664+0000","last_clean":"2024-09-14T06:59:52.487664+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T06:59:52.487664+0000","last_undegraded":"2024-09-14T06:59:52.487664+0000","last_fullsized":"2024-09-14T06:59:52.487664+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:17:20.223956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010420550000000001,"stat_sum":{"num_bytes":18259785,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":3497,"num_read_kb":6098,"num_write":107,"num_write_kb":2082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"40'91","reported_seq":2190,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-14T06:59:51.335589+0000","last_change":"2024-09-14T06:59:30.801946+0000","last_active":"2024-09-14T06:59:51.335589+0000","last_peered":"2024-09-14T06:59:51.335589+0000","last_clean":"2024-09-14T06:59:51.335589+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T06:59:51.335589+0000","last_undegraded":"2024-09-14T06:59:51.335589+0000","last_fullsized":"2024-09-14T06:59:51.335589+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:27:47.858971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081998900000000003,"stat_sum":{"num_bytes":47141388,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":2101,"num_read_kb":5763,"num_write":67,"num_write_kb":4612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"40'48","reported_seq":128,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.381573+0000","last_change":"2024-09-14T06:59:37.046075+0000","last_active":"2024-09-14T06:59:52.381573+0000","last_peered":"2024-09-14T06:59:52.381573+0000","last_clean":"2024-09-14T06:59:52.381573+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T06:59:52.381573+0000","last_undegraded":"2024-09-14T06:59:52.381573+0000","last_fullsized":"2024-09-14T06:59:52.381573+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:25.703998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059456000000000005,"stat_sum":{"num_bytes":5396381,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":52,"num_read_kb":2232,"num_write":38,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"46'69","reported_seq":1306,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T06:59:55.479806+0000","last_change":"2024-09-14T06:59:30.809133+0000","last_active":"2024-09-14T06:59:55.479806+0000","last_peered":"2024-09-14T06:59:55.479806+0000","last_clean":"2024-09-14T06:59:55.479806+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T06:59:55.479806+0000","last_undegraded":"2024-09-14T06:59:55.479806+0000","last_fullsized":"2024-09-14T06:59:55.479806+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:25:24.547747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055309100000000002,"stat_sum":{"num_bytes":4981683,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":11,"num_read":1655,"num_read_kb":5117,"num_write":52,"num_write_kb":188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"40'24","reported_seq":2565,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.326012+0000","last_change":"2024-09-14T06:59:30.803754+0000","last_active":"2024-09-14T06:59:52.326012+0000","last_peered":"2024-09-14T06:59:52.326012+0000","last_clean":"2024-09-14T06:59:52.326012+0000","last_became_active":"2024-09-14T06:59:28.801458+0000","last_became_peered":"2024-09-14T06:59:28.801458+0000","last_unstale":"2024-09-14T06:59:52.326012+0000","last_undegraded":"2024-09-14T06:59:52.326012+0000","last_fullsized":"2024-09-14T06:59:52.326012+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:11:33.421533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086036299999999995,"stat_sum":{"num_bytes":4497924,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":2485,"num_read_kb":5090,"num_write":25,"num_write_kb":1803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"40'37","reported_seq":147,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.381623+0000","last_change":"2024-09-14T06:59:51.355900+0000","last_active":"2024-09-14T06:59:52.381623+0000","last_peered":"2024-09-14T06:59:52.381623+0000","last_clean":"2024-09-14T06:59:52.381623+0000","last_became_active":"2024-09-14T06:59:51.330515+0000","last_became_peered":"2024-09-14T06:59:51.330515+0000","last_unstale":"2024-09-14T06:59:52.381623+0000","last_undegraded":"2024-09-14T06:59:52.381623+0000","last_fullsized":"2024-09-14T06:59:52.381623+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:33.741224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052297400000000005,"stat_sum":{"num_bytes":4320718,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":13,"num_read_kb":158,"num_write":37,"num_write_kb":1823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"43'116","reported_seq":2923,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.532772+0000","last_change":"2024-09-14T06:59:30.802873+0000","last_active":"2024-09-14T06:59:52.532772+0000","last_peered":"2024-09-14T06:59:52.532772+0000","last_clean":"2024-09-14T06:59:52.532772+0000","last_became_active":"2024-09-14T06:59:28.801310+0000","last_became_peered":"2024-09-14T06:59:28.801310+0000","last_unstale":"2024-09-14T06:59:52.532772+0000","last_undegraded":"2024-09-14T06:59:52.532772+0000","last_fullsized":"2024-09-14T06:59:52.532772+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:25:35.407267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013224059999999999,"stat_sum":{"num_bytes":14893249,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":2996,"num_read_kb":5913,"num_write":124,"num_write_kb":2473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.326100+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T06:59:52.326100+0000","last_peered":"2024-09-14T06:59:52.326100+0000","last_clean":"2024-09-14T06:59:52.326100+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T06:59:52.326100+0000","last_undegraded":"2024-09-14T06:59:52.326100+0000","last_fullsized":"2024-09-14T06:59:52.326100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"40'119","reported_seq":473,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-14T06:59:51.335188+0000","last_change":"2024-09-14T06:59:30.801713+0000","last_active":"2024-09-14T06:59:51.335188+0000","last_peered":"2024-09-14T06:59:51.335188+0000","last_clean":"2024-09-14T06:59:51.335188+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T06:59:51.335188+0000","last_undegraded":"2024-09-14T06:59:51.335188+0000","last_fullsized":"2024-09-14T06:59:51.335188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:17:28.931662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059990300000000005,"stat_sum":{"num_bytes":8430492,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":5,"num_read":1061,"num_read_kb":1199,"num_write":132,"num_write_kb":1276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-14T06:59:51.335307+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T06:59:51.335307+0000","last_peered":"2024-09-14T06:59:51.335307+0000","last_clean":"2024-09-14T06:59:51.335307+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T06:59:51.335307+0000","last_undegraded":"2024-09-14T06:59:51.335307+0000","last_fullsized":"2024-09-14T06:59:51.335307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"40'86","reported_seq":2422,"reported_epoch":42,"state":"activating","last_fresh":"2024-09-14T06:59:51.333025+0000","last_change":"2024-09-14T06:59:51.333598+0000","last_active":"2024-09-14T06:59:50.328836+0000","last_peered":"2024-09-14T06:59:50.269192+0000","last_clean":"2024-09-14T06:59:50.269192+0000","last_became_active":"2024-09-14T06:59:28.803870+0000","last_became_peered":"2024-09-14T06:59:28.803870+0000","last_unstale":"2024-09-14T06:59:51.333025+0000","last_undegraded":"2024-09-14T06:59:51.333025+0000","last_fullsized":"2024-09-14T06:59:51.333025+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060862000000000004,"stat_sum":{"num_bytes":4159105,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":3030,"num_read_kb":5714,"num_write":78,"num_write_kb":2073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4","objects":13}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.325898+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T06:59:52.325898+0000","last_peered":"2024-09-14T06:59:52.325898+0000","last_clean":"2024-09-14T06:59:52.325898+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T06:59:52.325898+0000","last_undegraded":"2024-09-14T06:59:52.325898+0000","last_fullsized":"2024-09-14T06:59:52.325898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"46'32","reported_seq":739,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T06:59:55.357217+0000","last_change":"2024-09-14T06:59:51.343911+0000","last_active":"2024-09-14T06:59:55.357217+0000","last_peered":"2024-09-14T06:59:55.357217+0000","last_clean":"2024-09-14T06:59:55.357217+0000","last_became_active":"2024-09-14T06:59:51.331353+0000","last_became_peered":"2024-09-14T06:59:51.331353+0000","last_unstale":"2024-09-14T06:59:55.357217+0000","last_undegraded":"2024-09-14T06:59:55.357217+0000","last_fullsized":"2024-09-14T06:59:55.357217+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:32.730413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1635942,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":392,"num_read_kb":240,"num_write":29,"num_write_kb":215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T06:59:55.357224+0000","last_change":"2024-09-14T06:59:51.331407+0000","last_active":"2024-09-14T06:59:55.357224+0000","last_peered":"2024-09-14T06:59:55.357224+0000","last_clean":"2024-09-14T06:59:55.357224+0000","last_became_active":"2024-09-14T06:59:51.331192+0000","last_became_peered":"2024-09-14T06:59:51.331192+0000","last_unstale":"2024-09-14T06:59:55.357224+0000","last_undegraded":"2024-09-14T06:59:55.357224+0000","last_fullsized":"2024-09-14T06:59:55.357224+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:41:42.693471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"38'98","reported_seq":3967,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.465227+0000","last_change":"2024-09-14T06:59:51.387715+0000","last_active":"2024-09-14T06:59:52.465227+0000","last_peered":"2024-09-14T06:59:52.465227+0000","last_clean":"2024-09-14T06:59:52.465227+0000","last_became_active":"2024-09-14T06:59:51.336748+0000","last_became_peered":"2024-09-14T06:59:51.336748+0000","last_unstale":"2024-09-14T06:59:52.465227+0000","last_undegraded":"2024-09-14T06:59:52.465227+0000","last_fullsized":"2024-09-14T06:59:52.465227+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:10:29.221947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052993500000000004,"stat_sum":{"num_bytes":24478816,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":11,"num_read":2203,"num_read_kb":4708,"num_write":83,"num_write_kb":3851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1805025,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"43'56","reported_seq":142,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.381758+0000","last_change":"2024-09-14T06:59:30.810757+0000","last_active":"2024-09-14T06:59:52.381758+0000","last_peered":"2024-09-14T06:59:52.381758+0000","last_clean":"2024-09-14T06:59:52.381758+0000","last_became_active":"2024-09-14T06:59:28.802025+0000","last_became_peered":"2024-09-14T06:59:28.802025+0000","last_unstale":"2024-09-14T06:59:52.381758+0000","last_undegraded":"2024-09-14T06:59:52.381758+0000","last_fullsized":"2024-09-14T06:59:52.381758+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:24:22.630291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044476100000000001,"stat_sum":{"num_bytes":12503299,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":8,"num_read":43,"num_read_kb":262,"num_write":64,"num_write_kb":3692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.325542+0000","last_change":"2024-09-14T06:59:51.752635+0000","last_active":"2024-09-14T06:59:52.325542+0000","last_peered":"2024-09-14T06:59:52.325542+0000","last_clean":"2024-09-14T06:59:52.325542+0000","last_became_active":"2024-09-14T06:59:51.749953+0000","last_became_peered":"2024-09-14T06:59:51.749953+0000","last_unstale":"2024-09-14T06:59:52.325542+0000","last_undegraded":"2024-09-14T06:59:52.325542+0000","last_fullsized":"2024-09-14T06:59:52.325542+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:31:47.116689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"46'74","reported_seq":6386,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-14T06:59:55.477506+0000","last_change":"2024-09-14T06:59:30.809288+0000","last_active":"2024-09-14T06:59:55.477506+0000","last_peered":"2024-09-14T06:59:55.477506+0000","last_clean":"2024-09-14T06:59:55.477506+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T06:59:55.477506+0000","last_undegraded":"2024-09-14T06:59:55.477506+0000","last_fullsized":"2024-09-14T06:59:55.477506+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:30.568293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063151899999999998,"stat_sum":{"num_bytes":9939901,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":6282,"num_read_kb":10486,"num_write":88,"num_write_kb":15183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-14T06:59:54.340073+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T06:59:54.340073+0000","last_peered":"2024-09-14T06:59:54.340073+0000","last_clean":"2024-09-14T06:59:54.340073+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T06:59:54.340073+0000","last_undegraded":"2024-09-14T06:59:54.340073+0000","last_fullsized":"2024-09-14T06:59:54.340073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"42'56","reported_seq":409,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.494097+0000","last_change":"2024-09-14T06:59:51.403760+0000","last_active":"2024-09-14T06:59:52.494097+0000","last_peered":"2024-09-14T06:59:52.494097+0000","last_clean":"2024-09-14T06:59:52.494097+0000","last_became_active":"2024-09-14T06:59:51.338478+0000","last_became_peered":"2024-09-14T06:59:51.338478+0000","last_unstale":"2024-09-14T06:59:52.494097+0000","last_undegraded":"2024-09-14T06:59:52.494097+0000","last_fullsized":"2024-09-14T06:59:52.494097+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:22:05.336702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2493004,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":5,"num_read":908,"num_read_kb":1125,"num_write":37,"num_write_kb":288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":270336,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.325241+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T06:59:52.325241+0000","last_peered":"2024-09-14T06:59:52.325241+0000","last_clean":"2024-09-14T06:59:52.325241+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T06:59:52.325241+0000","last_undegraded":"2024-09-14T06:59:52.325241+0000","last_fullsized":"2024-09-14T06:59:52.325241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"41'218","reported_seq":6291,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-14T06:59:51.335305+0000","last_change":"2024-09-14T06:59:30.801433+0000","last_active":"2024-09-14T06:59:51.335305+0000","last_peered":"2024-09-14T06:59:51.335305+0000","last_clean":"2024-09-14T06:59:51.335305+0000","last_became_active":"2024-09-14T06:59:28.801659+0000","last_became_peered":"2024-09-14T06:59:28.801659+0000","last_unstale":"2024-09-14T06:59:51.335305+0000","last_undegraded":"2024-09-14T06:59:51.335305+0000","last_fullsized":"2024-09-14T06:59:51.335305+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:00:09.243041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037135599999999999,"stat_sum":{"num_bytes":35306142,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":6986,"num_read_kb":28800,"num_write":369,"num_write_kb":9521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":63,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.325702+0000","last_change":"2024-09-14T06:59:21.767446+0000","last_active":"2024-09-14T06:59:52.325702+0000","last_peered":"2024-09-14T06:59:52.325702+0000","last_clean":"2024-09-14T06:59:52.325702+0000","last_became_active":"2024-09-14T06:59:20.120193+0000","last_became_peered":"2024-09-14T06:59:20.120193+0000","last_unstale":"2024-09-14T06:59:52.325702+0000","last_undegraded":"2024-09-14T06:59:52.325702+0000","last_fullsized":"2024-09-14T06:59:52.325702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:18.122142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.330703+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T06:59:52.330703+0000","last_peered":"2024-09-14T06:59:52.330703+0000","last_clean":"2024-09-14T06:59:52.330703+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T06:59:52.330703+0000","last_undegraded":"2024-09-14T06:59:52.330703+0000","last_fullsized":"2024-09-14T06:59:52.330703+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"40'90","reported_seq":2082,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-14T06:59:51.332530+0000","last_change":"2024-09-14T06:59:30.801426+0000","last_active":"2024-09-14T06:59:51.332530+0000","last_peered":"2024-09-14T06:59:51.332530+0000","last_clean":"2024-09-14T06:59:51.332530+0000","last_became_active":"2024-09-14T06:59:28.808414+0000","last_became_peered":"2024-09-14T06:59:28.808414+0000","last_unstale":"2024-09-14T06:59:51.332530+0000","last_undegraded":"2024-09-14T06:59:51.332530+0000","last_fullsized":"2024-09-14T06:59:51.332530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:33.623992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038565499999999999,"stat_sum":{"num_bytes":5546037,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":2186,"num_read_kb":6837,"num_write":126,"num_write_kb":4044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"43'163","reported_seq":1827,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.438049+0000","last_change":"2024-09-14T06:59:30.802594+0000","last_active":"2024-09-14T06:59:52.438049+0000","last_peered":"2024-09-14T06:59:52.438049+0000","last_clean":"2024-09-14T06:59:52.438049+0000","last_became_active":"2024-09-14T06:59:28.804973+0000","last_became_peered":"2024-09-14T06:59:28.804973+0000","last_unstale":"2024-09-14T06:59:52.438049+0000","last_undegraded":"2024-09-14T06:59:52.438049+0000","last_fullsized":"2024-09-14T06:59:52.438049+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:40:08.450601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793000000000001,"stat_sum":{"num_bytes":3014656,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":3122,"num_read_kb":3353,"num_write":152,"num_write_kb":783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"43'196","reported_seq":283,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.394790+0000","last_change":"2024-09-14T06:59:30.802721+0000","last_active":"2024-09-14T06:59:52.394790+0000","last_peered":"2024-09-14T06:59:52.394790+0000","last_clean":"2024-09-14T06:59:52.394790+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T06:59:52.394790+0000","last_undegraded":"2024-09-14T06:59:52.394790+0000","last_fullsized":"2024-09-14T06:59:52.394790+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:37:20.995866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402691,"stat_sum":{"num_bytes":2010659,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":781,"num_read_kb":1477,"num_write":296,"num_write_kb":1124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"45'75","reported_seq":1471,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-14T06:59:54.410582+0000","last_change":"2024-09-14T06:59:51.385585+0000","last_active":"2024-09-14T06:59:54.410582+0000","last_peered":"2024-09-14T06:59:54.410582+0000","last_clean":"2024-09-14T06:59:54.410582+0000","last_became_active":"2024-09-14T06:59:51.333054+0000","last_became_peered":"2024-09-14T06:59:51.333054+0000","last_unstale":"2024-09-14T06:59:54.410582+0000","last_undegraded":"2024-09-14T06:59:54.410582+0000","last_fullsized":"2024-09-14T06:59:54.410582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:22:07.010280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056983800000000003,"stat_sum":{"num_bytes":33896294,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":7,"num_read":1196,"num_read_kb":5855,"num_write":65,"num_write_kb":1466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":522922,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"40'35","reported_seq":2372,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.381603+0000","last_change":"2024-09-14T06:59:30.811244+0000","last_active":"2024-09-14T06:59:52.381603+0000","last_peered":"2024-09-14T06:59:52.381603+0000","last_clean":"2024-09-14T06:59:52.381603+0000","last_became_active":"2024-09-14T06:59:28.799619+0000","last_became_peered":"2024-09-14T06:59:28.799619+0000","last_unstale":"2024-09-14T06:59:52.381603+0000","last_undegraded":"2024-09-14T06:59:52.381603+0000","last_fullsized":"2024-09-14T06:59:52.381603+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:52:37.681564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077164499999999995,"stat_sum":{"num_bytes":3629144,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":6,"num_read":2291,"num_read_kb":4891,"num_write":33,"num_write_kb":1748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"40'134","reported_seq":274,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.499727+0000","last_change":"2024-09-14T06:59:30.803608+0000","last_active":"2024-09-14T06:59:52.499727+0000","last_peered":"2024-09-14T06:59:52.499727+0000","last_clean":"2024-09-14T06:59:52.499727+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T06:59:52.499727+0000","last_undegraded":"2024-09-14T06:59:52.499727+0000","last_fullsized":"2024-09-14T06:59:52.499727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:50:43.892710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016847640000000001,"stat_sum":{"num_bytes":13081010,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":467,"num_read_kb":2745,"num_write":188,"num_write_kb":4663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"43'78","reported_seq":5245,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.326223+0000","last_change":"2024-09-14T06:59:30.803975+0000","last_active":"2024-09-14T06:59:52.326223+0000","last_peered":"2024-09-14T06:59:52.326223+0000","last_clean":"2024-09-14T06:59:52.326223+0000","last_became_active":"2024-09-14T06:59:28.801111+0000","last_became_peered":"2024-09-14T06:59:28.801111+0000","last_unstale":"2024-09-14T06:59:52.326223+0000","last_undegraded":"2024-09-14T06:59:52.326223+0000","last_fullsized":"2024-09-14T06:59:52.326223+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:16:20.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084307800000000003,"stat_sum":{"num_bytes":41420377,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":7,"num_read":5410,"num_read_kb":27377,"num_write":83,"num_write_kb":10005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"38'27","reported_seq":2407,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.381726+0000","last_change":"2024-09-14T06:59:30.811224+0000","last_active":"2024-09-14T06:59:52.381726+0000","last_peered":"2024-09-14T06:59:52.381726+0000","last_clean":"2024-09-14T06:59:52.381726+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T06:59:52.381726+0000","last_undegraded":"2024-09-14T06:59:52.381726+0000","last_fullsized":"2024-09-14T06:59:52.381726+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:49:03.149347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053083799999999995,"stat_sum":{"num_bytes":5584961,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":5,"num_read":2335,"num_read_kb":5198,"num_write":26,"num_write_kb":1731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"42'117","reported_seq":2448,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.501293+0000","last_change":"2024-09-14T06:59:30.806812+0000","last_active":"2024-09-14T06:59:52.501293+0000","last_peered":"2024-09-14T06:59:52.501293+0000","last_clean":"2024-09-14T06:59:52.501293+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T06:59:52.501293+0000","last_undegraded":"2024-09-14T06:59:52.501293+0000","last_fullsized":"2024-09-14T06:59:52.501293+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:50:06.649653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055264400000000003,"stat_sum":{"num_bytes":9373599,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":2607,"num_read_kb":8590,"num_write":142,"num_write_kb":4283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"43'173","reported_seq":3461,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.549389+0000","last_change":"2024-09-14T06:59:30.802301+0000","last_active":"2024-09-14T06:59:52.549389+0000","last_peered":"2024-09-14T06:59:52.549389+0000","last_clean":"2024-09-14T06:59:52.549389+0000","last_became_active":"2024-09-14T06:59:28.801454+0000","last_became_peered":"2024-09-14T06:59:28.801454+0000","last_unstale":"2024-09-14T06:59:52.549389+0000","last_undegraded":"2024-09-14T06:59:52.549389+0000","last_fullsized":"2024-09-14T06:59:52.549389+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:24:42.583147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010054669999999999,"stat_sum":{"num_bytes":11577593,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":3961,"num_read_kb":6589,"num_write":272,"num_write_kb":6244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"43'125","reported_seq":702,"reported_epoch":43,"state":"active+undersized+remapped","last_fresh":"2024-09-14T06:59:52.435070+0000","last_change":"2024-09-14T06:59:52.435070+0000","last_active":"2024-09-14T06:59:52.435070+0000","last_peered":"2024-09-14T06:59:52.435070+0000","last_clean":"2024-09-14T06:59:49.319450+0000","last_became_active":"2024-09-14T06:59:52.414953+0000","last_became_peered":"2024-09-14T06:59:52.414953+0000","last_unstale":"2024-09-14T06:59:52.435070+0000","last_undegraded":"2024-09-14T06:59:52.435070+0000","last_fullsized":"2024-09-14T06:59:52.325703+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000465725,"stat_sum":{"num_bytes":28519286,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":1048,"num_read_kb":1158,"num_write":105,"num_write_kb":2311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":292050,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3],"avail_no_missing":["3","6","7"],"object_location_counts":[{"shards":"3","objects":11}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"43'103","reported_seq":427,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-14T06:59:52.325642+0000","last_change":"2024-09-14T06:59:30.802121+0000","last_active":"2024-09-14T06:59:52.325642+0000","last_peered":"2024-09-14T06:59:52.325642+0000","last_clean":"2024-09-14T06:59:52.325642+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T06:59:52.325642+0000","last_undegraded":"2024-09-14T06:59:52.325642+0000","last_fullsized":"2024-09-14T06:59:52.325642+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:14:40.741434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087321199999999995,"stat_sum":{"num_bytes":22784104,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":343,"num_read_kb":867,"num_write":62,"num_write_kb":972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":438024280,"num_objects":494,"num_object_clones":107,"num_object_copies":988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":494,"num_whiteouts":208,"num_read":69106,"num_read_kb":187727,"num_write":3317,"num_write_kb":113166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":3048385,"num_keys_recovered":23,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14282752,"data_stored":133644853,"data_compressed":4213986,"data_compressed_allocated":11214848,"data_compressed_original":131972956,"omap_allocated":0,"internal_metadata":0},"log_size":2964,"ondisk_log_size":2964,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":0,"seq":64424509452,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37580,"kb_used_data":2232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334260,"statfs":{"total":96636764160,"available":96598282240,"internally_reserved":0,"allocated":2285568,"data_stored":15437217,"data_compressed":571615,"data_compressed_allocated":1355776,"data_compressed_original":14977844,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37360,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334480,"statfs":{"total":96636764160,"available":96598507520,"internally_reserved":0,"allocated":2060288,"data_stored":11665296,"data_compressed":471332,"data_compressed_allocated":1208320,"data_compressed_original":11254068,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37772,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334068,"statfs":{"total":96636764160,"available":96598085632,"internally_reserved":0,"allocated":2473984,"data_stored":22395957,"data_compressed":554874,"data_compressed_allocated":1675264,"data_compressed_original":22026136,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33700,"kb_used_data":2512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338140,"statfs":{"total":96636764160,"available":96602255360,"internally_reserved":0,"allocated":2572288,"data_stored":15445676,"data_compressed":761182,"data_compressed_allocated":1556480,"data_compressed_original":14934016,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33864,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337976,"statfs":{"total":96636764160,"available":96602087424,"internally_reserved":0,"allocated":2744320,"data_stored":27595950,"data_compressed":570421,"data_compressed_allocated":1798144,"data_compressed_original":27139963,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33380,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338460,"statfs":{"total":96636764160,"available":96602583040,"internally_reserved":0,"allocated":2240512,"data_stored":14797691,"data_compressed":487271,"data_compressed_allocated":1368064,"data_compressed_original":14388736,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":1984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2031616,"data_stored":15895569,"data_compressed":405245,"data_compressed_allocated":1191936,"data_compressed_original":15496673,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2195456,"data_stored":15361305,"data_compressed":822512,"data_compressed_allocated":1622016,"data_compressed_original":15032320,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2142208,"data_stored":23523073,"data_compressed":584472,"data_compressed_allocated":1798144,"data_compressed_original":23357336,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1978368,"data_stored":15730536,"data_compressed":587480,"data_compressed_allocated":1576960,"data_compressed_original":15509812,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2162688,"data_stored":19480563,"data_compressed":649543,"data_compressed_allocated":1695744,"data_compressed_original":19217204,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1773568,"data_stored":14600312,"data_compressed":487271,"data_compressed_allocated":1368064,"data_compressed_original":14388736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2183168,"data_stored":26789955,"data_compressed":565290,"data_compressed_allocated":1761280,"data_compressed_original":26566523,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":310283,"data_compressed":48933,"data_compressed_allocated":81920,"data_compressed_original":241664,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1662976,"data_stored":17848826,"data_compressed":468485,"data_compressed_allocated":1310720,"data_compressed_original":17659361,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:59:56.146 INFO:tasks.thrashosds.thrasher:Setting 3.1f to [5, 0, 3, 2] 2024-09-14T06:59:56.146 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1f', '5', '0', '3', '2'] 2024-09-14T06:59:56.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1f 5 0 3 2 2024-09-14T06:59:56.173 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:56.173 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:56.181 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:59:56.223 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:56.222+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:56.306 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:56.305+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:56.325 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:56.325 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:56.334 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:59:56.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:59:56.406 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:56.405+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:56.469 INFO:teuthology.orchestra.run.smithi007.stdout:leaving image image_client.0-clone13 intact 2024-09-14T06:59:56.481 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:56.481 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:56.490 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:59:56.534 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:56.533+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:56.594 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:56.594 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:56.602 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:59:56.644 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:56.643+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:56.699 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-14T06:59:56.710 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:56.709+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:56.780 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:56.781 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:56.791 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:59:56.810 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:56.809+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:56.911 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:56.911+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:56.956 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:56.956 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:56.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:59:57.012 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:57.011+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:57.113 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:57.112+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:57.121 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:59:57.133 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:57.133 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:57.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:59:57.185 INFO:teuthology.orchestra.run.smithi007.stdout:1450 write 0xa6a9413 thru 0xa6acfe9 (0x3bd7 bytes) 2024-09-14T06:59:57.194 INFO:teuthology.orchestra.run.smithi007.stdout:1453 read 0x49877d3 thru 0x498be78 (0x46a6 bytes) 2024-09-14T06:59:57.195 INFO:teuthology.orchestra.run.smithi007.stdout:1456 punch from 0x8b9ead4 to 0x8ba4095, (0x55c1 bytes) 2024-09-14T06:59:57.195 INFO:teuthology.orchestra.run.smithi007.stdout:1462 write 0x6862475 thru 0x686fe82 (0xda0e bytes) 2024-09-14T06:59:57.203 INFO:teuthology.orchestra.run.smithi007.stdout:1463 write 0x77f812d thru 0x7801d5c (0x9c30 bytes) 2024-09-14T06:59:57.208 INFO:teuthology.orchestra.run.smithi007.stdout:1464 read 0x7033dba thru 0x7039d0c (0x5f53 bytes) 2024-09-14T06:59:57.213 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:57.213+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:57.219 INFO:teuthology.orchestra.run.smithi007.stdout:1465 trunc from 0xe13f4ec to 0x55c77fd 2024-09-14T06:59:57.261 INFO:teuthology.orchestra.run.smithi007.stdout:1466 write 0x9feb50c thru 0x9ff5729 (0xa21e bytes) 2024-09-14T06:59:57.267 INFO:teuthology.orchestra.run.smithi007.stdout:1467 read 0x7d8314 thru 0x7e264f (0xa33c bytes) 2024-09-14T06:59:57.272 INFO:teuthology.orchestra.run.smithi007.stdout:1468 trunc from 0x9ff572a to 0x8482e11 2024-09-14T06:59:57.282 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:57.282 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:57.283 INFO:teuthology.orchestra.run.smithi007.stdout:1469 punch from 0x51b5a5c to 0x51bd109, (0x76ad bytes) 2024-09-14T06:59:57.283 INFO:teuthology.orchestra.run.smithi007.stdout:1470 writesame 0x2ccf54b thru 0x2cd266e data_size 0x25(0x3124 bytes) 2024-09-14T06:59:57.289 INFO:teuthology.orchestra.run.smithi007.stdout:1471 write 0x5c393f8 thru 0x5c482b7 (0xeec0 bytes) 2024-09-14T06:59:57.296 INFO:teuthology.orchestra.run.smithi007.stdout:1472 writesame 0x6dc096a thru 0x6dcddc6 data_size 0x53(0xd45d bytes) 2024-09-14T06:59:57.308 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:57.311 INFO:teuthology.orchestra.run.smithi007.stdout:1473 write 0x9b73c3b thru 0x9b7b24e (0x7614 bytes) 2024-09-14T06:59:57.315 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:57.314+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:57.323 INFO:teuthology.orchestra.run.smithi007.stdout:1474 write 0xad4aeb1 thru 0xad4c5d2 (0x1722 bytes) 2024-09-14T06:59:57.325 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:57.325 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:57.333 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:57.335 INFO:teuthology.orchestra.run.smithi007.stdout:1476 write 0xca01bdd thru 0xca03ad3 (0x1ef7 bytes) 2024-09-14T06:59:57.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:59:57.339 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:57.339 INFO:teuthology.orchestra.run.smithi007.stdout:1478 read 0x3754262 thru 0x375923e (0x4fdd bytes) 2024-09-14T06:59:57.339 INFO:teuthology.orchestra.run.smithi007.stdout:1479 read 0x4ea1c62 thru 0x4eb064e (0xe9ed bytes) 2024-09-14T06:59:57.339 INFO:teuthology.orchestra.run.smithi007.stdout:1480 write 0x3f343e thru 0x402f9b (0xfb5e bytes) 2024-09-14T06:59:57.357 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:57.357 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.1f pg_upmap_items mapping to [5->0,3->2] 2024-09-14T06:59:57.365 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T06:59:57.388 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:57.388 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:57.403 INFO:teuthology.orchestra.run.smithi007.stdout:1481 write 0x7f7707 thru 0x7f8974 (0x126e bytes) 2024-09-14T06:59:57.415 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:57.415 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:57.415 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:57.414+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:57.434 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:57.434 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:57.451 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:57.452 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:57.475 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:57.475 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40676 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 761856~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210659+0000", 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.2636488240000006, 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0085896489999999995, 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40676 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210659+0000", 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210659+0000", 2024-09-14T06:59:57.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210660+0000", 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210662+0000", 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.796e-06 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210668+0000", 2024-09-14T06:59:57.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2430000000000002e-06 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210673+0000", 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8099999999999997e-06 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219087+0000", 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084145279999999992 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219156+0000", 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9277999999999996e-05 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219249+0000", 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2592999999999999e-05 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40678 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 835584~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.2636095170000008, 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0087422799999999998, 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40678 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210699+0000", 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210701+0000", 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.81e-06 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210707+0000", 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0260000000000004e-06 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210712+0000", 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:59:57.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219281+0000", 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008569587 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219351+0000", 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0055999999999995e-05 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219441+0000", 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9608e-05 2024-09-14T06:59:57.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40705 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 393216~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.2635502469999995, 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099565890000000001, 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40705 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210758+0000", 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210760+0000", 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210768+0000", 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9899999999999997e-06 2024-09-14T06:59:57.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210773+0000", 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0200000000000002e-06 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219470+0000", 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0086970339999999993 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219538+0000", 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7550000000000002e-05 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220715+0000", 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001176564 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40706 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 466944~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.2635282500000002, 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010139230000000001, 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40706 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210780+0000", 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210782+0000", 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210788+0000", 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1480000000000003e-06 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210793+0000", 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220747+0000", 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099533769999999994 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220813+0000", 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5926999999999994e-05 2024-09-14T06:59:57.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220919+0000", 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010641099999999999 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40718 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 385024~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.2635084699999997, 2024-09-14T06:59:57.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083164229999999999, 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40718 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210800+0000", 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210802+0000", 2024-09-14T06:59:57.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.886e-06 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210808+0000", 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0589999999999999e-06 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210815+0000", 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.218961+0000", 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081464520000000002 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219029+0000", 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7178e-05 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219116+0000", 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7569000000000001e-05 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40719 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 458752~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210822+0000", 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.2634860620000001, 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008481723, 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40719 2024-09-14T06:59:57.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210822+0000", 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210822+0000", 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210823+0000", 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210824+0000", 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.796e-06 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210836+0000", 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1567e-05 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210841+0000", 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3159999999999996e-06 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219153+0000", 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083116370000000002 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219220+0000", 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6958000000000005e-05 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219304+0000", 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4037999999999997e-05 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40739 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 679936~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210849+0000", 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.2634589560000009, 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008625789, 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40739 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210849+0000", 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210849+0000", 2024-09-14T06:59:57.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210850+0000", 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210852+0000", 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.153e-06 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210858+0000", 2024-09-14T06:59:57.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4470000000000003e-06 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210863+0000", 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9490000000000004e-06 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219333+0000", 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084702609999999998 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219398+0000", 2024-09-14T06:59:57.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4968999999999994e-05 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219475+0000", 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6625000000000006e-05 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40740 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 753664~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210869+0000", 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.2634387819999997, 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088293469999999995, 2024-09-14T06:59:57.507 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40740 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210869+0000", 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210869+0000", 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210870+0000", 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210886+0000", 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6568000000000001e-05 2024-09-14T06:59:57.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210893+0000", 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8759999999999999e-06 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210899+0000", 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.648e-06 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219503+0000", 2024-09-14T06:59:57.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0086043809999999995 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219612+0000", 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109064 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219699+0000", 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6419e-05 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40741 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 827392~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.2634013119999992, 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0089720200000000007, 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40741 2024-09-14T06:59:57.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210907+0000", 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.75e-07 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210909+0000", 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210916+0000", 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1659999999999999e-06 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.210920+0000", 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8640000000000004e-06 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219728+0000", 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088070700000000002 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219794+0000", 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6014999999999997e-05 2024-09-14T06:59:57.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.219879+0000", 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5370000000000004e-05 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:48248 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 179200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.738998+0000", 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.735310363, 2024-09-14T06:59:57.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083283609999999994, 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 48248 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738998+0000", 2024-09-14T06:59:57.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738998+0000", 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738998+0000", 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739000+0000", 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0650000000000001e-06 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739006+0000", 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739011+0000", 2024-09-14T06:59:57.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.747191+0000", 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081793260000000007 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.747252+0000", 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1674000000000006e-05 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.747326+0000", 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3912000000000007e-05 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53209 3.c 3:3661da64:::rbd_data.114f54fbf0ed.0000000000000000:9 [read 851968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.042252+0000", 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.4320564769999997, 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091733390000000008, 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53209 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042252+0000", 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042252+0000", 2024-09-14T06:59:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042252+0000", 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2799999999999996e-07 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042255+0000", 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042264+0000", 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9700000000000005e-06 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.042270+0000", 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051175+0000", 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0089050499999999994 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051284+0000", 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109417 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051425+0000", 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000140994 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53384 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.050817+0000", 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.4234912680000003, 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008397247, 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53384 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050817+0000", 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050817+0000", 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050817+0000", 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:59:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050819+0000", 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0810000000000001e-06 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050826+0000", 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5390000000000004e-06 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050836+0000", 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0529e-05 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.058431+0000", 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075943019999999998 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.058500+0000", 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9558e-05 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.059214+0000", 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00071383600000000003 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53424 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 886784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.050960+0000", 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.4233486290000004, 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014641082999999999, 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53424 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050960+0000", 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050960+0000", 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050960+0000", 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050962+0000", 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1009999999999999e-06 2024-09-14T06:59:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050969+0000", 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7630000000000003e-06 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.050974+0000", 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9389999999999996e-06 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.059248+0000", 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082738120000000002 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.059318+0000", 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0680999999999996e-05 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.065601+0000", 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0062823769999999996 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53425 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 889856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.051001+0000", 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.4233075560000001, 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098477459999999992, 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53425 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051001+0000", 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051001+0000", 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051001+0000", 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9800000000000004e-07 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051004+0000", 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.446e-06 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051011+0000", 2024-09-14T06:59:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7659999999999998e-06 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051017+0000", 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1e-06 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060637+0000", 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096191520000000006 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060709+0000", 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2407e-05 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060848+0000", 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013937699999999999 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53462 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 884736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.051100+0000", 2024-09-14T06:59:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.4232083800000002, 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099824709999999997, 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53462 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051100+0000", 2024-09-14T06:59:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051100+0000", 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051100+0000", 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.34e-07 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051103+0000", 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051110+0000", 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0890000000000002e-06 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051115+0000", 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.592e-06 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060877+0000", 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097612459999999995 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.060948+0000", 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1293999999999999e-05 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.061082+0000", 2024-09-14T06:59:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013445199999999999 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53463 3.f 3:f6af4224:::rbd_data.114f54fbf0ed.0000000000000005:9 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.051123+0000", 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.4231850369999997, 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010210110999999999, 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53463 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051123+0000", 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051123+0000", 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051124+0000", 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051126+0000", 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2879999999999999e-06 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051133+0000", 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8940000000000004e-06 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.051138+0000", 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4809999999999997e-06 2024-09-14T06:59:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.061109+0000", 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099709050000000004 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.061179+0000", 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0236999999999999e-05 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.061333+0000", 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015389900000000001 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53763 3.18 3:1a0a685e:::rbd_data.114f8f5a97df.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.209787+0000", 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.2645211249999999, 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010489207, 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53763 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.209787+0000", 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.209787+0000", 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.209789+0000", 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.844e-06 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210712+0000", 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000923524 2024-09-14T06:59:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210730+0000", 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7793000000000001e-05 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210739+0000", 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6200000000000005e-06 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210817+0000", 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:59:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210952+0000", 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013575600000000001 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.211554+0000", 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000601181 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.218261+0000", 2024-09-14T06:59:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076719999999996 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220167+0000", 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001905389 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220229+0000", 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2769999999999997e-05 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220276+0000", 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6869999999999997e-05 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:53764 3.18 3:1a0a685e:::rbd_data.114f8f5a97df.0000000000000008:head [truncate 524288] snapc c=[c] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:50.210764+0000", 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.2635440579999999, 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097383399999999998, 2024-09-14T06:59:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 53764 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210764+0000", 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210764+0000", 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210765+0000", 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.282e-06 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210769+0000", 2024-09-14T06:59:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1559999999999999e-06 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210779+0000", 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0874e-05 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.210785+0000", 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4280000000000004e-06 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.216013+0000", 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052282400000000003 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.216116+0000", 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010279499999999999 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.216306+0000", 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018991 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.218694+0000", 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023877770000000002 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220409+0000", 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001715531 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220461+0000", 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1653000000000002e-05 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:50.220502+0000", 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1693999999999998e-05 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:62460 3.3 3:c626242c:::rbd_data.114ffe9c58f5.000000000000007c:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:52.932764+0000", 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5415443880000002, 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008344888, 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 62460 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932764+0000", 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932764+0000", 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932765+0000", 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.305e-06 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932771+0000", 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0000000000000002e-06 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932783+0000", 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1429e-05 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932788+0000", 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5779999999999998e-06 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932848+0000", 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.995e-05 2024-09-14T06:59:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.932924+0000", 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5828999999999995e-05 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.934533+0000", 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00160866 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.941041+0000", 2024-09-14T06:59:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065081820000000004 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.941050+0000", 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5200000000000003e-06 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.941080+0000", 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0209000000000002e-05 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:52.941109+0000", 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8226e-05 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:63728 3.f 3:f2ae5d6d:::rbd_data.114f95439a0c.0000000000000211:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:54.667728+0000", 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.8065801910000001, 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082406579999999997, 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 63728 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.667728+0000", 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.667728+0000", 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.667729+0000", 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8899999999999998e-07 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.667733+0000", 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2810000000000002e-06 2024-09-14T06:59:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.667744+0000", 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0865e-05 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.667752+0000", 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8790000000000003e-06 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.667810+0000", 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8230000000000003e-05 2024-09-14T06:59:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.667937+0000", 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126546 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.669613+0000", 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00167639 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.675811+0000", 2024-09-14T06:59:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006197691 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.675864+0000", 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3238000000000001e-05 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.675925+0000", 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0532999999999998e-05 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:54.675969+0000", 2024-09-14T06:59:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4016e-05 2024-09-14T06:59:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.573 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:57.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:59:57.575 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:57.574+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:57.675 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:57.675+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:57.722 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:57.723 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:57.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:59:57.776 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:57.776+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:57.838 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:57.838 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:57.838 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:57.838 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:57.838 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:57.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:59:57.890 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:57.890+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:57.956 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:59:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.220671170000003, 2024-09-14T06:59:57.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:57.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T06:59:57.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17618 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.236329+0000", 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.718058232000001, 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016649290000000001, 2024-09-14T06:59:57.963 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17618 2024-09-14T06:59:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236332+0000", 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237113+0000", 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078082900000000003 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237131+0000", 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7747000000000001e-05 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237141+0000", 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0137e-05 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237221+0000", 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0482000000000006e-05 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237383+0000", 2024-09-14T06:59:57.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016176100000000001 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237737+0000", 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035380999999999998 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.241638+0000", 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039011739999999999 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252854+0000", 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011216097 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252919+0000", 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5129999999999995e-05 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252979+0000", 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9573e-05 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40284 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:9 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.204456+0000", 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.7499311710000001, 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016005640000000002, 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40284 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204457+0000", 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204459+0000", 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204466+0000", 2024-09-14T06:59:57.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8730000000000001e-06 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204472+0000", 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4779999999999999e-06 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207358+0000", 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028862549999999999 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207430+0000", 2024-09-14T06:59:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2268000000000007e-05 2024-09-14T06:59:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220462+0000", 2024-09-14T06:59:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013031951999999999 2024-09-14T06:59:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40646 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.7487895370000004, 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017879991000000001, 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40646 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205605+0000", 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2260000000000002e-06 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205611+0000", 2024-09-14T06:59:57.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3289999999999999e-06 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205622+0000", 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213726+0000", 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008104745 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213792+0000", 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5401999999999995e-05 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223478+0000", 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096862100000000007 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40683 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:57.979 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.7487176180000006, 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016494009, 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40683 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:57.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205672+0000", 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0379999999999998e-06 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205679+0000", 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205684+0000", 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9359999999999998e-06 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221845+0000", 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016161422000000002 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221913+0000", 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7669000000000006e-05 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222164+0000", 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025130600000000002 2024-09-14T06:59:57.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40685 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205690+0000", 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.7486972139999999, 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016657634000000001, 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40685 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205691+0000", 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205693+0000", 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205699+0000", 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205703+0000", 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5240000000000002e-06 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222190+0000", 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016486752 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222257+0000", 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7095999999999996e-05 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222348+0000", 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1061000000000001e-05 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40687 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.7486779840000004, 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016826543999999999, 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40687 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:59:57.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205712+0000", 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205718+0000", 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205723+0000", 2024-09-14T06:59:57.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222374+0000", 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016650933999999999 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222445+0000", 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1304999999999996e-05 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222536+0000", 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1104000000000001e-05 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40720 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205826+0000", 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.7485613880000006, 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016893371000000001, 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40720 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205827+0000", 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.58e-07 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205834+0000", 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2729999999999997e-06 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205841+0000", 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8260000000000004e-06 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205846+0000", 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1769999999999997e-06 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222562+0000", 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016716178000000002 2024-09-14T06:59:57.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222633+0000", 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0461000000000002e-05 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222720+0000", 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6997999999999996e-05 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40721 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.748533686, 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044822000000001, 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40721 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:57.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205857+0000", 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-06 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205863+0000", 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205868+0000", 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T06:59:58.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222745+0000", 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016877293000000002 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222814+0000", 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8806000000000005e-05 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222899+0000", 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4913000000000005e-05 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40742 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.7485131200000001, 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017201659000000001, 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40742 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:58.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205892+0000", 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7008e-05 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205899+0000", 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1489999999999997e-06 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205904+0000", 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222923+0000", 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017018229999999999 2024-09-14T06:59:58.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222991+0000", 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8545000000000002e-05 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223076+0000", 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4938999999999994e-05 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40743 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205913+0000", 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.748474388, 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343654999999999, 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40743 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.007 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205914+0000", 2024-09-14T06:59:58.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-07 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205916+0000", 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0559999999999999e-06 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205922+0000", 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3659999999999997e-06 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205927+0000", 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8169999999999998e-06 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223101+0000", 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017174452 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223168+0000", 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6756999999999998e-05 2024-09-14T06:59:58.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223257+0000", 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8700000000000001e-05 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43363 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5013006220000005, 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015032403, 2024-09-14T06:59:58.011 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43363 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453089+0000", 2024-09-14T06:59:58.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9980000000000002e-06 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453096+0000", 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3060000000000003e-06 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453101+0000", 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8960000000000002e-06 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465082+0000", 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011981083 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465162+0000", 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0588999999999997e-05 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468119+0000", 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002957079 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43452 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5010329319999993, 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014487193000000001, 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43452 2024-09-14T06:59:58.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3299999999999998e-07 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453357+0000", 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8500000000000001e-06 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453363+0000", 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3239999999999999e-06 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453368+0000", 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2290000000000002e-06 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467645+0000", 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014276812999999999 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467720+0000", 2024-09-14T06:59:58.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4492000000000004e-05 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467842+0000", 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012215199999999999 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43453 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 78848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5010127210000004, 2024-09-14T06:59:58.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015012878, 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43453 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453377+0000", 2024-09-14T06:59:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1050000000000002e-06 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453384+0000", 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453389+0000", 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-06 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467869+0000", 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014479845999999999 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467944+0000", 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5680999999999996e-05 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468388+0000", 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000443473 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43576 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 99328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4985798559999992, 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014560714000000001, 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43576 2024-09-14T06:59:58.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455809+0000", 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.091e-06 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455812+0000", 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199999999999999e-06 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455815+0000", 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.537e-06 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470167+0000", 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014352746 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470245+0000", 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7730000000000003e-05 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470368+0000", 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012338 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.029 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43577 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 102400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.498569775, 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014789722999999999, 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43577 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455820+0000", 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4270000000000001e-06 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455823+0000", 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0139999999999999e-06 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455825+0000", 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4980000000000001e-06 2024-09-14T06:59:58.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470402+0000", 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014576858999999999 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470485+0000", 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2731000000000005e-05 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470608+0000", 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012297299999999999 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43578 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4985491189999998, 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015008412, 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43578 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4400000000000001e-07 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455841+0000", 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2749999999999998e-06 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455848+0000", 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9349999999999997e-06 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455854+0000", 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-06 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470639+0000", 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014785382999999999 2024-09-14T06:59:58.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470716+0000", 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7155000000000002e-05 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470847+0000", 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013058000000000001 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43579 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4985257589999996, 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01522916, 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43579 2024-09-14T06:59:58.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455864+0000", 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455870+0000", 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T06:59:58.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455875+0000", 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9509999999999997e-06 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470879+0000", 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015003729 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470960+0000", 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1228000000000002e-05 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471091+0000", 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131018 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43580 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 110592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455881+0000", 2024-09-14T06:59:58.043 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4985062209999995, 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015447047, 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43580 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455882+0000", 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455884+0000", 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2010000000000002e-06 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455891+0000", 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5690000000000001e-06 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455896+0000", 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3639999999999998e-06 2024-09-14T06:59:58.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471123+0000", 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015226805 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471200+0000", 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7405999999999997e-05 2024-09-14T06:59:58.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471328+0000", 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012835099999999999 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43581 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455903+0000", 2024-09-14T06:59:58.048 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4984843199999993, 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015668949000000001, 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43581 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455904+0000", 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455910+0000", 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6619999999999999e-06 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455917+0000", 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5769999999999999e-06 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455923+0000", 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6779999999999997e-06 2024-09-14T06:59:58.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471360+0000", 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015436854999999999 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471443+0000", 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3815999999999998e-05 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471572+0000", 2024-09-14T06:59:58.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012892900000000001 2024-09-14T06:59:58.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.053 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:58.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:59:58.055 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:58.054+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:58.155 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:58.155+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:58.202 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:58.202 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:58.202 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:58.210 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:59:58.256 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:58.256+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:58.357 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:58.356+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:58.358 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:58.358 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:58.358 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:58.358 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:58.358 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:58.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:59:58.458 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:58.458+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:58.516 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:15473 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.124194+0000", 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.390947239999999, 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052649906000000003, 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15473 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124194+0000", 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124194+0000", 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124195+0000", 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2700000000000001e-07 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124196+0000", 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1650000000000001e-06 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124199+0000", 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.269e-06 2024-09-14T06:59:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124202+0000", 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6000000000000001e-06 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176600+0000", 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052398307999999998 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176668+0000", 2024-09-14T06:59:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7859000000000002e-05 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176844+0000", 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017647800000000001 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:15474 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.124205+0000", 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.390936719000001, 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053002028999999999, 2024-09-14T06:59:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15474 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124205+0000", 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124205+0000", 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130009+0000", 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058039299999999997 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130013+0000", 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7129999999999999e-06 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130021+0000", 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7749999999999998e-06 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130026+0000", 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.938e-06 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176891+0000", 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046864380999999997 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176972+0000", 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1581000000000004e-05 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.177207+0000", 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000234711 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18567 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 12926976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319102+0000", 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.196039746, 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052720059999999999, 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18567 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319102+0000", 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319102+0000", 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319102+0000", 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319104+0000", 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.356e-06 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319107+0000", 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6040000000000001e-06 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319110+0000", 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8729999999999999e-06 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371714+0000", 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052603556000000003 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371770+0000", 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6097999999999997e-05 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371822+0000", 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2207999999999999e-05 2024-09-14T06:59:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18568 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13041664~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319114+0000", 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.196028074999999, 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052952595999999998, 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18568 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319114+0000", 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319114+0000", 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319114+0000", 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319115+0000", 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319118+0000", 2024-09-14T06:59:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0599999999999999e-06 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319121+0000", 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7319999999999998e-06 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371870+0000", 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052749273999999999 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371959+0000", 2024-09-14T06:59:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9061000000000007e-05 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372066+0000", 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107299 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18569 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13156352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319124+0000", 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.196017617000001, 2024-09-14T06:59:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053177681999999997, 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18569 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319124+0000", 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319124+0000", 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319124+0000", 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319125+0000", 2024-09-14T06:59:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319129+0000", 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9910000000000004e-06 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319132+0000", 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8119999999999999e-06 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372102+0000", 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052970060999999999 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372192+0000", 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9778999999999995e-05 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372302+0000", 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001099 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18570 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13271040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319135+0000", 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.196006333, 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053394978000000003, 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18570 2024-09-14T06:59:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319135+0000", 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319135+0000", 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319136+0000", 2024-09-14T06:59:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2399999999999999e-07 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319137+0000", 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.206e-06 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319140+0000", 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1159999999999999e-06 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319143+0000", 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6800000000000002e-06 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372337+0000", 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194033000000002 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372424+0000", 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7880000000000006e-05 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372530+0000", 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105839 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18571 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13385728~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319148+0000", 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.195993184000001, 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053601401999999999, 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18571 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319148+0000", 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319148+0000", 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319149+0000", 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319150+0000", 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2160000000000001e-06 2024-09-14T06:59:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319155+0000", 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0520000000000001e-06 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319159+0000", 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3329999999999998e-06 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372564+0000", 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053404576000000002 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372652+0000", 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7719000000000005e-05 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372750+0000", 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8256000000000003e-05 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18572 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13500416~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319165+0000", 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.195976748, 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053777864000000002, 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18572 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319165+0000", 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319165+0000", 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319165+0000", 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319166+0000", 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.172e-06 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319170+0000", 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3589999999999999e-06 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319172+0000", 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6740000000000001e-06 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372792+0000", 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053620062000000003 2024-09-14T06:59:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372883+0000", 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0847000000000006e-05 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372943+0000", 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9469000000000003e-05 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18573 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13615104~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319175+0000", 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.195966158999999, 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054002965, 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18573 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319175+0000", 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319175+0000", 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319176+0000", 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:59:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319177+0000", 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319182+0000", 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0919999999999997e-06 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319186+0000", 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:59:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372988+0000", 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053801579000000002 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373077+0000", 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.954e-05 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373178+0000", 2024-09-14T06:59:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101271 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18574 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13729792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319192+0000", 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.195949448, 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054130926000000003, 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18574 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319192+0000", 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319192+0000", 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319192+0000", 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319194+0000", 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319199+0000", 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2909999999999998e-06 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319203+0000", 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5469999999999998e-06 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373210+0000", 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054006709999999999 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373270+0000", 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0229999999999998e-05 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373323+0000", 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2675e-05 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18575 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13844480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319210+0000", 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.195931713, 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054350181999999997, 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18575 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319210+0000", 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319210+0000", 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319210+0000", 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7500000000000001e-07 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319212+0000", 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0640000000000001e-06 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319219+0000", 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2489999999999998e-06 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319224+0000", 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0170000000000004e-06 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373364+0000", 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054140677999999998 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373452+0000", 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7497000000000006e-05 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373560+0000", 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010830199999999999 2024-09-14T06:59:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18576 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13959168~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319231+0000", 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.195910962999999, 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054486288000000001, 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18576 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319231+0000", 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319231+0000", 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319231+0000", 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319233+0000", 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0439999999999998e-06 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319237+0000", 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7560000000000001e-06 2024-09-14T06:59:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319240+0000", 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373591+0000", 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054351337999999999 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373647+0000", 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5300000000000002e-05 2024-09-14T06:59:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373717+0000", 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0276000000000003e-05 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18577 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 14073856~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319265+0000", 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.195876396999999, 2024-09-14T06:59:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054723081999999999, 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18577 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319265+0000", 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319265+0000", 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319265+0000", 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319267+0000", 2024-09-14T06:59:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319271+0000", 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3200000000000001e-06 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319274+0000", 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1499999999999999e-06 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373772+0000", 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054497800999999998 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373883+0000", 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110654 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373988+0000", 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105404 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18578 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 14188544~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319279+0000", 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.195862891000001, 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054840223, 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18578 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319279+0000", 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319279+0000", 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324912+0000", 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056328250000000002 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324916+0000", 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0330000000000002e-06 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324926+0000", 2024-09-14T06:59:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0379e-05 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324931+0000", 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374008+0000", 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.049076501000000002 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374065+0000", 2024-09-14T06:59:58.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6753999999999999e-05 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374119+0000", 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4441999999999998e-05 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18645 3.16 3:6af7eeae:::rbd_data.114f54fbf0ed.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.324991+0000", 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.190151044, 2024-09-14T06:59:58.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054731618000000003, 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18645 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324991+0000", 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324991+0000", 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324991+0000", 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9700000000000003e-07 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326138+0000", 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011472909999999999 2024-09-14T06:59:58.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326147+0000", 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8089999999999999e-06 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326152+0000", 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374799+0000", 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.048647060999999998 2024-09-14T06:59:58.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374946+0000", 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014760700000000001 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.375283+0000", 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033679499999999998 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379203+0000", 2024-09-14T06:59:58.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039194779999999997 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379627+0000", 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042489399999999999 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379683+0000", 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5037999999999997e-05 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379722+0000", 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9682999999999999e-05 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18809 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 10985472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326645+0000", 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.188496711999999, 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052693862000000001, 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18809 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326645+0000", 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326645+0000", 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326645+0000", 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326651+0000", 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2129999999999996e-06 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326656+0000", 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8860000000000003e-06 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326659+0000", 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379202+0000", 2024-09-14T06:59:58.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052542356999999998 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379280+0000", 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8115999999999995e-05 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379339+0000", 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9104999999999997e-05 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18810 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11100160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326663+0000", 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.188478525000001, 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052824810999999999, 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18810 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.590 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326663+0000", 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326663+0000", 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326663+0000", 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-14T06:59:58.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326664+0000", 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1179999999999999e-06 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326669+0000", 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7600000000000002e-06 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326674+0000", 2024-09-14T06:59:58.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3499999999999999e-06 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379369+0000", 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052695288999999999 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379427+0000", 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8074e-05 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379488+0000", 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0977999999999998e-05 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18811 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11214848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326680+0000", 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.188461808, 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053046902, 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18811 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326680+0000", 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326680+0000", 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326680+0000", 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5400000000000002e-07 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326682+0000", 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9429999999999999e-06 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326688+0000", 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6269999999999997e-06 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326691+0000", 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8990000000000001e-06 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379514+0000", 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052823399 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379559+0000", 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4613000000000001e-05 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379727+0000", 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016806699999999999 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18812 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11329536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326694+0000", 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.188447160000001, 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053113457000000003, 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18812 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326694+0000", 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326694+0000", 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326695+0000", 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2600000000000001e-07 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326696+0000", 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1999999999999999e-06 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326699+0000", 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.101e-06 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326702+0000", 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7630000000000001e-06 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379684+0000", 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052982143000000002 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379756+0000", 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2056000000000003e-05 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379808+0000", 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1968000000000001e-05 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.602 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18813 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11444224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326705+0000", 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.188436808000001, 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053264406, 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:58.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18813 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326705+0000", 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326705+0000", 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:58.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331872+0000", 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051673459999999997 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331877+0000", 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5460000000000002e-06 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331887+0000", 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0078000000000001e-05 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331892+0000", 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379878+0000", 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047986046999999997 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379919+0000", 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:58.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:58.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:58.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379969+0000", 2024-09-14T06:59:58.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0185999999999997e-05 2024-09-14T06:59:58.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:58.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:58.607 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:58.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:59:58.609 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:58.608+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:58.709 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:58.708+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:58.757 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:58.757 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:58.758 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T06:59:58.758 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:58.766 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:59:58.850 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T06:59:58.849+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T06:59:58.910 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:58.910+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:58.913 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T06:59:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T06:59:58.913 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T06:59:58.913 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:58.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:59:59.011 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T06:59:59.011+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T06:59:59.074 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T06:59:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T06:59:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T06:59:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T06:59:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T06:59:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.315796483, 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T06:59:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T06:59:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T06:59:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.717383891000001, 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T06:59:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T06:59:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T06:59:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T06:59:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.711724247999999, 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T06:59:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T06:59:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4058879700000002, 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T06:59:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T06:59:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4058571779999998, 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T06:59:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4058338080000006, 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T06:59:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4050272990000003, 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T06:59:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4050049060000003, 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T06:59:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4049847660000001, 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T06:59:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4048346110000001, 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T06:59:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4047790570000007, 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T06:59:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T06:59:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T06:59:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4047460160000007, 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T06:59:59.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.133 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4023929309999996, 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T06:59:59.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4023703659999995, 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45774 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 933888~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668061+0000", 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4015336289999993, 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068647313000000001, 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45774 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T06:59:59.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668062+0000", 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668064+0000", 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.058e-06 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668070+0000", 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668075+0000", 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7859999999999996e-06 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736553+0000", 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068478400999999994 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736622+0000", 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8732999999999994e-05 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736709+0000", 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6453999999999996e-05 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4015133710000001, 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.151 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.151 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T06:59:59.151 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4002963499999996, 2024-09-14T06:59:59.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T06:59:59.151 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4001685100000003, 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4001484170000005, 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.164 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.3991215239999999, 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T06:59:59.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T06:59:59.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T06:59:59.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T06:59:59.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T06:59:59.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T06:59:59.169 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T06:59:59.170 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:59:59.171 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:59.170+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:59.271 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T06:59:59.270+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T06:59:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:59.286 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:59.286 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T06:59:59.286 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:59.295 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:59:59.379 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:59.379+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:59.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:59:59.442 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:59.442 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T06:59:59.442 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T06:59:59.442 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T06:59:59.442 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:59.451 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:59:59.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:59:59.492 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:59.491+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:59.572 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T06:59:59.572+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T06:59:59.591 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T06:59:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:59:59.596 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T06:59:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T06:59:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T06:59:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T06:59:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7619 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25436160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533626+0000", 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.061423286, 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050835010999999999, 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.597 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7619 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533626+0000", 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533626+0000", 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533627+0000", 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9499999999999998e-07 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533629+0000", 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.812e-06 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533634+0000", 2024-09-14T06:59:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9529999999999996e-06 2024-09-14T06:59:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533639+0000", 2024-09-14T06:59:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6999999999999999e-06 2024-09-14T06:59:59.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.583252+0000", 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049613288999999998 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.583328+0000", 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5112999999999997e-05 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584461+0000", 2024-09-14T06:59:59.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0011338489999999999 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7620 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25550848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533671+0000", 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.061378288, 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050947723, 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.602 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7620 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533671+0000", 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533671+0000", 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533672+0000", 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9800000000000003e-07 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533675+0000", 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0139999999999999e-06 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533686+0000", 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1104000000000001e-05 2024-09-14T06:59:59.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533691+0000", 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9189999999999998e-06 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584498+0000", 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050806905999999999 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584547+0000", 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8733999999999998e-05 2024-09-14T06:59:59.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584619+0000", 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2447999999999995e-05 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7621 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25665536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533699+0000", 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.061350208, 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051134163000000003, 2024-09-14T06:59:59.606 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7621 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533699+0000", 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533699+0000", 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533700+0000", 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533701+0000", 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4950000000000001e-06 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533705+0000", 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.788e-06 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533709+0000", 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8419999999999998e-06 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584659+0000", 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050950016000000001 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584747+0000", 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8117999999999999e-05 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584834+0000", 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6500000000000002e-05 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.610 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7622 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25780224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533713+0000", 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.061336548, 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051311927, 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7622 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533713+0000", 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533713+0000", 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533713+0000", 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.29e-07 2024-09-14T06:59:59.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533716+0000", 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.362e-06 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533719+0000", 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3129999999999999e-06 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533722+0000", 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6259999999999999e-06 2024-09-14T06:59:59.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584874+0000", 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051152345000000002 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584960+0000", 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5795000000000001e-05 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585025+0000", 2024-09-14T06:59:59.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5257000000000002e-05 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7623 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25894912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533725+0000", 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.061324581000001, 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051497312000000003, 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.615 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7623 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533725+0000", 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533725+0000", 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533725+0000", 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533726+0000", 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533729+0000", 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.01e-06 2024-09-14T06:59:59.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533732+0000", 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.57e-06 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585061+0000", 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051328632999999999 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585144+0000", 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3061999999999999e-05 2024-09-14T06:59:59.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585222+0000", 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8794999999999994e-05 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7624 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26009600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533735+0000", 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.061314798000002, 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051626949999999998, 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.619 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7624 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533735+0000", 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533735+0000", 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533913+0000", 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017811300000000001 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533918+0000", 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6820000000000002e-06 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533932+0000", 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3899e-05 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533939+0000", 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5190000000000003e-06 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585252+0000", 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051313304999999997 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585297+0000", 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4149e-05 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585362+0000", 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5283000000000005e-05 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.623 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7625 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26124288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533952+0000", 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.061097846999999, 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051605167, 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7625 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533952+0000", 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533952+0000", 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534144+0000", 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019176499999999999 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534147+0000", 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.382e-06 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534156+0000", 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2289999999999992e-06 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534161+0000", 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585402+0000", 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051240648 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585484+0000", 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1897000000000005e-05 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585557+0000", 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3449999999999996e-05 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.627 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7626 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26238976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.534171+0000", 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.060878402, 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051588410000000001, 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7626 2024-09-14T06:59:59.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534171+0000", 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534171+0000", 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534297+0000", 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012619899999999999 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534305+0000", 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3329999999999999e-06 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534313+0000", 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4290000000000001e-06 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534320+0000", 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6780000000000003e-06 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585598+0000", 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051277572 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585682+0000", 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4858000000000006e-05 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585760+0000", 2024-09-14T06:59:59.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7341000000000004e-05 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7627 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26353664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.534330+0000", 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.060719303999999, 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051630638, 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.632 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7627 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534330+0000", 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534330+0000", 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534576+0000", 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00024544699999999998 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534581+0000", 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7870000000000001e-06 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534589+0000", 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6119999999999999e-06 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534596+0000", 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6590000000000001e-06 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585801+0000", 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051204972000000001 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585883+0000", 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2494e-05 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585961+0000", 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7667e-05 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7628 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26468352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.534609+0000", 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.060441179000001, 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051474580999999998, 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.636 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7628 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534609+0000", 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534609+0000", 2024-09-14T06:59:59.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538869+0000", 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042609450000000004 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538874+0000", 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5050000000000001e-06 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538884+0000", 2024-09-14T06:59:59.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0242e-05 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538891+0000", 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2119999999999997e-06 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585991+0000", 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.047099540000000002 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.586038+0000", 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6971000000000001e-05 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.586083+0000", 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5166000000000001e-05 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7644 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28319744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539166+0000", 2024-09-14T06:59:59.640 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.055883274999999, 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049804237000000001, 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7644 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539166+0000", 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539166+0000", 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539167+0000", 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539168+0000", 2024-09-14T06:59:59.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7100000000000011e-07 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539172+0000", 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0980000000000004e-06 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539175+0000", 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0319999999999999e-06 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.588864+0000", 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049688831000000003 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.588928+0000", 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4193e-05 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.588971+0000", 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2910000000000001e-05 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7645 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28434432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539179+0000", 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.055870272, 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04998959, 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7645 2024-09-14T06:59:59.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539179+0000", 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539179+0000", 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539180+0000", 2024-09-14T06:59:59.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539182+0000", 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4229999999999999e-06 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539189+0000", 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6270000000000002e-06 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539194+0000", 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4970000000000001e-06 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589004+0000", 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049809785000000002 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589091+0000", 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7348999999999993e-05 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589169+0000", 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7655000000000001e-05 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7646 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28549120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539202+0000", 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.055847985, 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050152584, 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.649 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7646 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539202+0000", 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539202+0000", 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539202+0000", 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6199999999999999e-07 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539203+0000", 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.218e-06 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539207+0000", 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2540000000000003e-06 2024-09-14T06:59:59.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539210+0000", 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.757e-06 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589199+0000", 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049988800999999999 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589275+0000", 2024-09-14T06:59:59.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6174000000000006e-05 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589354+0000", 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9117999999999997e-05 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7647 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28663808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539216+0000", 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.055833269000001, 2024-09-14T06:59:59.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050330617000000001, 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7647 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539216+0000", 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539216+0000", 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539217+0000", 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539218+0000", 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6589999999999999e-06 2024-09-14T06:59:59.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539225+0000", 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2859999999999996e-06 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539229+0000", 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3069999999999996e-06 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589391+0000", 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0501623 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589472+0000", 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0752000000000002e-05 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589547+0000", 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5018999999999994e-05 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7648 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28778496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539236+0000", 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.055814000000002, 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050463816000000002, 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7648 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.658 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539236+0000", 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539236+0000", 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539236+0000", 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T06:59:59.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539237+0000", 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.263e-06 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539241+0000", 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3960000000000002e-06 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539243+0000", 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7269999999999999e-06 2024-09-14T06:59:59.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589586+0000", 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050343176000000003 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589656+0000", 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9530999999999995e-05 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589700+0000", 2024-09-14T06:59:59.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3511999999999999e-05 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7649 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28893184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539247+0000", 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.055803129000001, 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050647407999999998, 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7649 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539247+0000", 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539247+0000", 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539247+0000", 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539248+0000", 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0279999999999999e-06 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539251+0000", 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539253+0000", 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2989999999999999e-06 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589733+0000", 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050480216000000001 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589817+0000", 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4159000000000005e-05 2024-09-14T06:59:59.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589894+0000", 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6595999999999996e-05 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7650 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 29007872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539256+0000", 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.055793924, 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050806128999999998, 2024-09-14T06:59:59.666 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7650 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539256+0000", 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539256+0000", 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539256+0000", 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9999999999999999e-07 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539261+0000", 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7820000000000001e-06 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539265+0000", 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1219999999999997e-06 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539267+0000", 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5689999999999999e-06 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589925+0000", 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050657695000000003 2024-09-14T06:59:59.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589979+0000", 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4088999999999997e-05 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590062+0000", 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2671999999999997e-05 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.670 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7651 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 29122560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539271+0000", 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.055778880999998, 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050987034000000001, 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7651 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539271+0000", 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539271+0000", 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539271+0000", 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539272+0000", 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4099999999999997e-07 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539275+0000", 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0699999999999998e-06 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539278+0000", 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5179999999999999e-06 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590100+0000", 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050822289 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590181+0000", 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1269999999999999e-05 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590258+0000", 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.674 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7652 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 29237248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539280+0000", 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.055769353999999, 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051152192999999999, 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7652 2024-09-14T06:59:59.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539280+0000", 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539280+0000", 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543135+0000", 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0038546919999999998 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543140+0000", 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7910000000000004e-06 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543151+0000", 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1314e-05 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543156+0000", 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9590000000000003e-06 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590297+0000", 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.047140973000000003 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590381+0000", 2024-09-14T06:59:59.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4006999999999997e-05 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590433+0000", 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1456999999999999e-05 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7654 3.2 3:41572fcc:::rbd_object_map.114f54fbf0ed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.543182+0000", 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 16.051867743999999, 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050091802999999997, 2024-09-14T06:59:59.679 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7654 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543182+0000", 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543182+0000", 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543182+0000", 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6600000000000002e-07 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543184+0000", 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.063e-06 2024-09-14T06:59:59.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543188+0000", 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8680000000000001e-06 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543191+0000", 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.881e-06 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590653+0000", 2024-09-14T06:59:59.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.047461934999999997 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590706+0000", 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3340000000000001e-05 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 2", 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.591041+0000", 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00033495900000000002 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593167+0000", 2024-09-14T06:59:59.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021259389999999999 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593183+0000", 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5515e-05 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593233+0000", 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9777000000000001e-05 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T06:59:59.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593274+0000", 2024-09-14T06:59:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1159999999999999e-05 2024-09-14T06:59:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T06:59:59.685 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T06:59:59.685 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T06:59:59.686 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T06:59:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:59:59.687 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T06:59:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:59:59.687 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:59:59.688 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T06:59:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:59:59.688 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T06:59:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:59:59.689 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T06:59:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:59:59.689 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T06:59:59.685+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T06:59:59.692 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T06:59:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:59:59.710 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T06:59:59.710 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:59:59.788 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T06:59:59.786+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T06:59:59.832 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:59.833 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:59.841 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:59:59.900 INFO:teuthology.orchestra.run.smithi007.stdout:1484 trunc from 0xca03ad4 to 0xaa861a2 2024-09-14T06:59:59.910 INFO:teuthology.orchestra.run.smithi007.stdout:1485 read 0x1826a1a thru 0x1827091 (0x678 bytes) 2024-09-14T06:59:59.910 INFO:teuthology.orchestra.run.smithi007.stdout:1487 writesame 0x66b812 thru 0x66bc4b data_size 0x1(0x43a bytes) 2024-09-14T06:59:59.912 INFO:teuthology.orchestra.run.smithi007.stdout:1489 punch from 0x41794be to 0x41809fd, (0x753f bytes) 2024-09-14T06:59:59.912 INFO:teuthology.orchestra.run.smithi007.stdout:1490 write 0x55a8bc3 thru 0x55a905c (0x49a bytes) 2024-09-14T06:59:59.921 INFO:teuthology.orchestra.run.smithi007.stdout:1491 write 0xd434060 thru 0xd437bc3 (0x3b64 bytes) 2024-09-14T06:59:59.924 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:59.925 INFO:teuthology.orchestra.run.smithi007.stdout:1494 punch from 0x5c832f2 to 0x5c92f02, (0xfc10 bytes) 2024-09-14T06:59:59.925 INFO:teuthology.orchestra.run.smithi007.stdout:1495 trunc from 0xd437bc4 to 0xdbbff85 2024-09-14T06:59:59.929 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T06:59:59.929+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T06:59:59.930 INFO:teuthology.orchestra.run.smithi007.stdout:1496 read 0xdba308e thru 0xdba38c2 (0x835 bytes) 2024-09-14T06:59:59.930 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T06:59:59.936 INFO:teuthology.orchestra.run.smithi007.stdout:1498 write 0xe636ce0 thru 0xe63b675 (0x4996 bytes) 2024-09-14T06:59:59.940 INFO:teuthology.orchestra.run.smithi007.stdout:1499 read 0x5aebf4c thru 0x5af28c4 (0x6979 bytes) 2024-09-14T06:59:59.941 INFO:teuthology.orchestra.run.smithi007.stdout:1501 read 0x2d53257 thru 0x2d5475d (0x1507 bytes) 2024-09-14T06:59:59.941 INFO:teuthology.orchestra.run.smithi007.stdout:1503 writesame 0xad0adc1 thru 0xad0d6a5 data_size 0x13(0x28e5 bytes) 2024-09-14T06:59:59.945 INFO:teuthology.orchestra.run.smithi007.stdout:1504 write 0x2914d8f thru 0x2916530 (0x17a2 bytes) 2024-09-14T06:59:59.949 INFO:teuthology.orchestra.run.smithi007.stdout:1505 clone 15 order 20 su 4096 sc 3 2024-09-14T06:59:59.988 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T06:59:59.988+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T06:59:59.989 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T06:59:59.989 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:59:59.998 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:00.089 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:00.089+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:00.145 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:00.145 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:00.154 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:00.190 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:00.190+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:00.291 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:00.291+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:00.298 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:00.298 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:00.307 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:00.381 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T07:00:00.392 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:00.391+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:00.453 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:00.453 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:00.462 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:00.503 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:00.502+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:00.593 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:00.593+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:00.606 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:00.606 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:00.615 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:00.700 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:00.699+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:00.759 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:00.759 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:00.769 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:00.806 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:00.802+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:00.897 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:00.896+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:00.916 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:00.916 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:00.924 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:00.998 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:00.998+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:01.089 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:01.089 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:01.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:01.098 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:01.098+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:01.199 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:01.199+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:01.220 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:01.220 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:01.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:01.300 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:01.300+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:01.338 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:01.338 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:01.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:01.401 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:01.400+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:01.446 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone14 from 0xe63b676 (overlap 0x55c77fd) to 0x893f32 2024-09-14T07:00:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:01.482 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:01.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:01.539 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:01.538+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:01.603 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:01.602+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:01.675 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:01.675 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:01.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:01.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:01.725 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:01.724+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:01.804 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:01.803+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:01.836 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:01.836 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:01.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:01.852 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:01.853 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:01.871 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:01.904 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:01.904+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:01.908 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:01.909 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:01.943 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:01.943 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:01.977 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:01.977 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:02.006 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:02.005+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:02.010 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:02.011 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:02.013 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:02.013 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.278311045999999, 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:00:02.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:00:02.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:02.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17618 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.775698108, 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016649290000000001, 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17618 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236332+0000", 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-14T07:00:02.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237113+0000", 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078082900000000003 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237131+0000", 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7747000000000001e-05 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237141+0000", 2024-09-14T07:00:02.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0137e-05 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237221+0000", 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0482000000000006e-05 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237383+0000", 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016176100000000001 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237737+0000", 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035380999999999998 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.241638+0000", 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039011739999999999 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252854+0000", 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011216097 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252919+0000", 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5129999999999995e-05 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252979+0000", 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9573e-05 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40284 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:9 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.807571047, 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016005640000000002, 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40284 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204457+0000", 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204459+0000", 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204466+0000", 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8730000000000001e-06 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204472+0000", 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4779999999999999e-06 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207358+0000", 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028862549999999999 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207430+0000", 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2268000000000007e-05 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220462+0000", 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013031951999999999 2024-09-14T07:00:02.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40646 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.806429413, 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017879991000000001, 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40646 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205605+0000", 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2260000000000002e-06 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205611+0000", 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3289999999999999e-06 2024-09-14T07:00:02.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205622+0000", 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213726+0000", 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008104745 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213792+0000", 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5401999999999995e-05 2024-09-14T07:00:02.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223478+0000", 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096862100000000007 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40683 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.806357494, 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016494009, 2024-09-14T07:00:02.035 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40683 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205672+0000", 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0379999999999998e-06 2024-09-14T07:00:02.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205679+0000", 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205684+0000", 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9359999999999998e-06 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221845+0000", 2024-09-14T07:00:02.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016161422000000002 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221913+0000", 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7669000000000006e-05 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222164+0000", 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025130600000000002 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40685 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.80633709, 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016657634000000001, 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40685 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205691+0000", 2024-09-14T07:00:02.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:00:02.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205693+0000", 2024-09-14T07:00:02.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-14T07:00:02.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205699+0000", 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205703+0000", 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5240000000000002e-06 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222190+0000", 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016486752 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222257+0000", 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7095999999999996e-05 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222348+0000", 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1061000000000001e-05 2024-09-14T07:00:02.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40687 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.80631786, 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016826543999999999, 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40687 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4999999999999998e-07 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205712+0000", 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205718+0000", 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205723+0000", 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222374+0000", 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016650933999999999 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222445+0000", 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1304999999999996e-05 2024-09-14T07:00:02.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222536+0000", 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1104000000000001e-05 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40720 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.806201264, 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016893371000000001, 2024-09-14T07:00:02.049 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40720 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205827+0000", 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.58e-07 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205834+0000", 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2729999999999997e-06 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205841+0000", 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205846+0000", 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1769999999999997e-06 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222562+0000", 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016716178000000002 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222633+0000", 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0461000000000002e-05 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222720+0000", 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6997999999999996e-05 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.053 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40721 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.806173562, 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044822000000001, 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40721 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205857+0000", 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-06 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205863+0000", 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205868+0000", 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222745+0000", 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016877293000000002 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222814+0000", 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8806000000000005e-05 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222899+0000", 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4913000000000005e-05 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40742 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.806152996, 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017201659000000001, 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40742 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:02.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205892+0000", 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7008e-05 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205899+0000", 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1489999999999997e-06 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205904+0000", 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222923+0000", 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017018229999999999 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222991+0000", 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8545000000000002e-05 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223076+0000", 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4938999999999994e-05 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40743 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.806114264, 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343654999999999, 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40743 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205914+0000", 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-07 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205916+0000", 2024-09-14T07:00:02.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0559999999999999e-06 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205922+0000", 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3659999999999997e-06 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205927+0000", 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8169999999999998e-06 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223101+0000", 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017174452 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223168+0000", 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6756999999999998e-05 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223257+0000", 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8700000000000001e-05 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43363 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.558940498, 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015032403, 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43363 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:00:02.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453089+0000", 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9980000000000002e-06 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453096+0000", 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3060000000000003e-06 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453101+0000", 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8960000000000002e-06 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465082+0000", 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011981083 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465162+0000", 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0588999999999997e-05 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468119+0000", 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002957079 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43452 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453355+0000", 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.558672808000001, 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014487193000000001, 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43452 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453355+0000", 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3299999999999998e-07 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453357+0000", 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8500000000000001e-06 2024-09-14T07:00:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453363+0000", 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3239999999999999e-06 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453368+0000", 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2290000000000002e-06 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467645+0000", 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014276812999999999 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467720+0000", 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4492000000000004e-05 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467842+0000", 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012215199999999999 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43453 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 78848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.558652597, 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015012878, 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43453 2024-09-14T07:00:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453377+0000", 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1050000000000002e-06 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453384+0000", 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453389+0000", 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-06 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467869+0000", 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014479845999999999 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467944+0000", 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5680999999999996e-05 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468388+0000", 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000443473 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43576 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 99328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455808+0000", 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.556219732000001, 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014560714000000001, 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43576 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T07:00:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455808+0000", 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455809+0000", 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.091e-06 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455812+0000", 2024-09-14T07:00:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199999999999999e-06 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455815+0000", 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.537e-06 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470167+0000", 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014352746 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470245+0000", 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7730000000000003e-05 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470368+0000", 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012338 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43577 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 102400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455818+0000", 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.556209651, 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014789722999999999, 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43577 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455818+0000", 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455820+0000", 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4270000000000001e-06 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455823+0000", 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0139999999999999e-06 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455825+0000", 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4980000000000001e-06 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470402+0000", 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014576858999999999 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470485+0000", 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2731000000000005e-05 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470608+0000", 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012297299999999999 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43578 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455839+0000", 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.556188994999999, 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015008412, 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43578 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455839+0000", 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4400000000000001e-07 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455841+0000", 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2749999999999998e-06 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455848+0000", 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9349999999999997e-06 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455854+0000", 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-06 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470639+0000", 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014785382999999999 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470716+0000", 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7155000000000002e-05 2024-09-14T07:00:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470847+0000", 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013058000000000001 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43579 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.556165634999999, 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01522916, 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43579 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455864+0000", 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455870+0000", 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455875+0000", 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9509999999999997e-06 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470879+0000", 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015003729 2024-09-14T07:00:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470960+0000", 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1228000000000002e-05 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471091+0000", 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131018 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43580 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 110592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.556146096999999, 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015447047, 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43580 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455882+0000", 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455884+0000", 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2010000000000002e-06 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455891+0000", 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5690000000000001e-06 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455896+0000", 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3639999999999998e-06 2024-09-14T07:00:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471123+0000", 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015226805 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471200+0000", 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7405999999999997e-05 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471328+0000", 2024-09-14T07:00:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012835099999999999 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43581 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.556124196000001, 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015668949000000001, 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43581 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455904+0000", 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455910+0000", 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6619999999999999e-06 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455917+0000", 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5769999999999999e-06 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455923+0000", 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6779999999999997e-06 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471360+0000", 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015436854999999999 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471443+0000", 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3815999999999998e-05 2024-09-14T07:00:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471572+0000", 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012892900000000001 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.106 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:02.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:02.108 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:02.108 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:02.109 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:02.109 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:02.110 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:02.107+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:02.209 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:02.208+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:02.263 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:02.263 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:02.263 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:02.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:02.289 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-14T07:00:02.311 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:02.309+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:02.393 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:02.393 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:02.393 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-14T07:00:02.393 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:00:02.394 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:00:02.394 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-14T07:00:02.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:00:02.429 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:02.429 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:02.429 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:02.429 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:02.428+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:02.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:02.531 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:02.530+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:02.531 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:02.531 INFO:teuthology.orchestra.run.smithi189.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T07:00:02.531 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:02.612 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:02.611+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:02.671 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:15473 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.124194+0000", 2024-09-14T07:00:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.544861529999999, 2024-09-14T07:00:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052649906000000003, 2024-09-14T07:00:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15473 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124194+0000", 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124194+0000", 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124195+0000", 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124196+0000", 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1650000000000001e-06 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124199+0000", 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.269e-06 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124202+0000", 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6000000000000001e-06 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176600+0000", 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052398307999999998 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176668+0000", 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7859000000000002e-05 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176844+0000", 2024-09-14T07:00:02.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017647800000000001 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:15474 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 2064384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.124205+0000", 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.544851008999998, 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053002028999999999, 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15474 2024-09-14T07:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124205+0000", 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.124205+0000", 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130009+0000", 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058039299999999997 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130013+0000", 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7129999999999999e-06 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130021+0000", 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7749999999999998e-06 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.130026+0000", 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.938e-06 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176891+0000", 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.046864380999999997 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.176972+0000", 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1581000000000004e-05 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.177207+0000", 2024-09-14T07:00:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000234711 2024-09-14T07:00:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18567 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 12926976~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319102+0000", 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349954036, 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052720059999999999, 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18567 2024-09-14T07:00:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319102+0000", 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319102+0000", 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319102+0000", 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319104+0000", 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.356e-06 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319107+0000", 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6040000000000001e-06 2024-09-14T07:00:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319110+0000", 2024-09-14T07:00:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8729999999999999e-06 2024-09-14T07:00:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371714+0000", 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052603556000000003 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371770+0000", 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6097999999999997e-05 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371822+0000", 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2207999999999999e-05 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18568 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13041664~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319114+0000", 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349942365, 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052952595999999998, 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18568 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319114+0000", 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319114+0000", 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319114+0000", 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319115+0000", 2024-09-14T07:00:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319118+0000", 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0599999999999999e-06 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319121+0000", 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7319999999999998e-06 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371870+0000", 2024-09-14T07:00:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052749273999999999 2024-09-14T07:00:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.371959+0000", 2024-09-14T07:00:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9061000000000007e-05 2024-09-14T07:00:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372066+0000", 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107299 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18569 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13156352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319124+0000", 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349931906999998, 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053177681999999997, 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18569 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319124+0000", 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319124+0000", 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319124+0000", 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319125+0000", 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2399999999999996e-07 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319129+0000", 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9910000000000004e-06 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319132+0000", 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8119999999999999e-06 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372102+0000", 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052970060999999999 2024-09-14T07:00:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372192+0000", 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9778999999999995e-05 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372302+0000", 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001099 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18570 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13271040~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319135+0000", 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349920622999999, 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053394978000000003, 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18570 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319135+0000", 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319135+0000", 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319136+0000", 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319137+0000", 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.206e-06 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319140+0000", 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1159999999999999e-06 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319143+0000", 2024-09-14T07:00:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6800000000000002e-06 2024-09-14T07:00:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372337+0000", 2024-09-14T07:00:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053194033000000002 2024-09-14T07:00:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372424+0000", 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7880000000000006e-05 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372530+0000", 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105839 2024-09-14T07:00:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18571 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13385728~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319148+0000", 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349907473999998, 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053601401999999999, 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18571 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319148+0000", 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319148+0000", 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319149+0000", 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319150+0000", 2024-09-14T07:00:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2160000000000001e-06 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319155+0000", 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0520000000000001e-06 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319159+0000", 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3329999999999998e-06 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372564+0000", 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053404576000000002 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372652+0000", 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7719000000000005e-05 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372750+0000", 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8256000000000003e-05 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18572 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13500416~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319165+0000", 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349891037999999, 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053777864000000002, 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18572 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319165+0000", 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319165+0000", 2024-09-14T07:00:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319165+0000", 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319166+0000", 2024-09-14T07:00:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.172e-06 2024-09-14T07:00:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319170+0000", 2024-09-14T07:00:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3589999999999999e-06 2024-09-14T07:00:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319172+0000", 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6740000000000001e-06 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372792+0000", 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053620062000000003 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372883+0000", 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0847000000000006e-05 2024-09-14T07:00:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372943+0000", 2024-09-14T07:00:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9469000000000003e-05 2024-09-14T07:00:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18573 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13615104~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319175+0000", 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349880449, 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054002965, 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18573 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319175+0000", 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319175+0000", 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319176+0000", 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319177+0000", 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9999999999999995e-07 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319182+0000", 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0919999999999997e-06 2024-09-14T07:00:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319186+0000", 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2629999999999997e-06 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.372988+0000", 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053801579000000002 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373077+0000", 2024-09-14T07:00:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.954e-05 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373178+0000", 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101271 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18574 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13729792~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319192+0000", 2024-09-14T07:00:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349863738, 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054130926000000003, 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18574 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319192+0000", 2024-09-14T07:00:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319192+0000", 2024-09-14T07:00:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319192+0000", 2024-09-14T07:00:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:00:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319194+0000", 2024-09-14T07:00:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.223e-06 2024-09-14T07:00:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319199+0000", 2024-09-14T07:00:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2909999999999998e-06 2024-09-14T07:00:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319203+0000", 2024-09-14T07:00:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5469999999999998e-06 2024-09-14T07:00:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373210+0000", 2024-09-14T07:00:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054006709999999999 2024-09-14T07:00:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373270+0000", 2024-09-14T07:00:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0229999999999998e-05 2024-09-14T07:00:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373323+0000", 2024-09-14T07:00:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2675e-05 2024-09-14T07:00:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18575 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13844480~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319210+0000", 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349846003, 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054350181999999997, 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18575 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319210+0000", 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319210+0000", 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319210+0000", 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7500000000000001e-07 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319212+0000", 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0640000000000001e-06 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319219+0000", 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2489999999999998e-06 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319224+0000", 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0170000000000004e-06 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373364+0000", 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054140677999999998 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373452+0000", 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7497000000000006e-05 2024-09-14T07:00:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373560+0000", 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010830199999999999 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18576 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 13959168~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319231+0000", 2024-09-14T07:00:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349825252999999, 2024-09-14T07:00:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054486288000000001, 2024-09-14T07:00:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18576 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319231+0000", 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319231+0000", 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319231+0000", 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319233+0000", 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0439999999999998e-06 2024-09-14T07:00:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319237+0000", 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7560000000000001e-06 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319240+0000", 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0709999999999999e-06 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373591+0000", 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054351337999999999 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373647+0000", 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5300000000000002e-05 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373717+0000", 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0276000000000003e-05 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18577 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 14073856~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319265+0000", 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349790686999999, 2024-09-14T07:00:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054723081999999999, 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18577 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319265+0000", 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319265+0000", 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319265+0000", 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6100000000000002e-07 2024-09-14T07:00:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319267+0000", 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319271+0000", 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3200000000000001e-06 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319274+0000", 2024-09-14T07:00:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1499999999999999e-06 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373772+0000", 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054497800999999998 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373883+0000", 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110654 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.373988+0000", 2024-09-14T07:00:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105404 2024-09-14T07:00:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18578 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 14188544~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.319279+0000", 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.349777181, 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054840223, 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18578 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319279+0000", 2024-09-14T07:00:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.319279+0000", 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324912+0000", 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056328250000000002 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324916+0000", 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0330000000000002e-06 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324926+0000", 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0379e-05 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324931+0000", 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374008+0000", 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.049076501000000002 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374065+0000", 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6753999999999999e-05 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374119+0000", 2024-09-14T07:00:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4441999999999998e-05 2024-09-14T07:00:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18645 3.16 3:6af7eeae:::rbd_data.114f54fbf0ed.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.324991+0000", 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.344065334, 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054731618000000003, 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18645 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324991+0000", 2024-09-14T07:00:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324991+0000", 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.324991+0000", 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9700000000000003e-07 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326138+0000", 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011472909999999999 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326147+0000", 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8089999999999999e-06 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326152+0000", 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6650000000000002e-06 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374799+0000", 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.048647060999999998 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.374946+0000", 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014760700000000001 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-14T07:00:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.375283+0000", 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033679499999999998 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379203+0000", 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039194779999999997 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379627+0000", 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00042489399999999999 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379683+0000", 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5037999999999997e-05 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379722+0000", 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9682999999999999e-05 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18809 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 10985472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326645+0000", 2024-09-14T07:00:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.342411001999999, 2024-09-14T07:00:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052693862000000001, 2024-09-14T07:00:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18809 2024-09-14T07:00:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326645+0000", 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326645+0000", 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326645+0000", 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326651+0000", 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2129999999999996e-06 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326656+0000", 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8860000000000003e-06 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326659+0000", 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9670000000000001e-06 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379202+0000", 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052542356999999998 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379280+0000", 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8115999999999995e-05 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379339+0000", 2024-09-14T07:00:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9104999999999997e-05 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18810 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11100160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326663+0000", 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.342392815, 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052824810999999999, 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18810 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326663+0000", 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326663+0000", 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326663+0000", 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000002e-07 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326664+0000", 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1179999999999999e-06 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326669+0000", 2024-09-14T07:00:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7600000000000002e-06 2024-09-14T07:00:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326674+0000", 2024-09-14T07:00:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3499999999999999e-06 2024-09-14T07:00:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379369+0000", 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052695288999999999 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379427+0000", 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8074e-05 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379488+0000", 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0977999999999998e-05 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18811 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11214848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326680+0000", 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.342376097999999, 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053046902, 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18811 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326680+0000", 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326680+0000", 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326680+0000", 2024-09-14T07:00:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:00:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326682+0000", 2024-09-14T07:00:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9429999999999999e-06 2024-09-14T07:00:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326688+0000", 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6269999999999997e-06 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326691+0000", 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8990000000000001e-06 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379514+0000", 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052823399 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379559+0000", 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4613000000000001e-05 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379727+0000", 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016806699999999999 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18812 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11329536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326694+0000", 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.342361449999999, 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053113457000000003, 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18812 2024-09-14T07:00:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326694+0000", 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326694+0000", 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326695+0000", 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2600000000000001e-07 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326696+0000", 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1999999999999999e-06 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326699+0000", 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.101e-06 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326702+0000", 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7630000000000001e-06 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379684+0000", 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.052982143000000002 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379756+0000", 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2056000000000003e-05 2024-09-14T07:00:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379808+0000", 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1968000000000001e-05 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:18813 3.e 3:7037416e:::rbd_data.114f7dc3575d.0000000000000001:8 [read 11444224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.326705+0000", 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.342351098000002, 2024-09-14T07:00:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053264406, 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 18813 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326705+0000", 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.326705+0000", 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331872+0000", 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051673459999999997 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331877+0000", 2024-09-14T07:00:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5460000000000002e-06 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331887+0000", 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0078000000000001e-05 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.331892+0000", 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379878+0000", 2024-09-14T07:00:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.047986046999999997 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379919+0000", 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1051999999999999e-05 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.379969+0000", 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0185999999999997e-05 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:02.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:02.783 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:02.781+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:02.883 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:02.882+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:02.935 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:02.935 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:02.935 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:02.935 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:02.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:02.983 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:02.983+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:03.009 INFO:teuthology.orchestra.run.smithi007.stdout:1507 write 0x1d7bf39 thru 0x1d7f4d9 (0x35a1 bytes) 2024-09-14T07:00:03.062 INFO:teuthology.orchestra.run.smithi007.stdout:1508 read 0xd54fd05 thru 0xd55280e (0x2b0a bytes) 2024-09-14T07:00:03.062 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:03.062 INFO:teuthology.orchestra.run.smithi007.stdout:1510 writesame 0x81a8365 thru 0x81b4a69 data_size 0x9(0xc705 bytes) 2024-09-14T07:00:03.076 INFO:teuthology.orchestra.run.smithi007.stdout:1512 trunc from 0xe63b676 to 0x8061c0d 2024-09-14T07:00:03.084 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:03.084+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:03.101 INFO:teuthology.orchestra.run.smithi007.stdout:1514 trunc from 0x8061c0d to 0xc6e86c 2024-09-14T07:00:03.130 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:03.130 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:03.130 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:03.130 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:03.130 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:03.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:03.185 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:03.185+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:03.286 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:03.286+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:03.296 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.541271223000003, 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:03.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:00:03.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:00:03.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.942858631, 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:03.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:00:03.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.937198987999999, 2024-09-14T07:00:03.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:00:03.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:00:03.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:00:03.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:03.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.631362709999999, 2024-09-14T07:00:03.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:00:03.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.631331918000001, 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:00:03.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:03.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.631308548, 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:03.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:03.331 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.630502039, 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:03.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:00:03.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.630479646, 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:00:03.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:00:03.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:00:03.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.630459505999999, 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:03.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:00:03.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:00:03.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.630309350999999, 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:00:03.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:00:03.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.630253797, 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:00:03.349 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:00:03.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.630220756, 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:00:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:00:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.627867671000001, 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:00:03.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:03.362 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.627845106000001, 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:03.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45774 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 933888~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.627008369, 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068647313000000001, 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45774 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668062+0000", 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668064+0000", 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.058e-06 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668070+0000", 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668075+0000", 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7859999999999996e-06 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736553+0000", 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068478400999999994 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736622+0000", 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8732999999999994e-05 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736709+0000", 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6453999999999996e-05 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.626988110999999, 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:00:03.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.375 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.625771090000001, 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.62564325, 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:00:03.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:00:03.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.625623157, 2024-09-14T07:00:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:00:03.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:00:03.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.624596264000001, 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.389 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:00:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:00:03.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:00:03.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:00:03.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:03.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:03.393 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:03.394 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:03.395 INFO:teuthology.orchestra.run.smithi007.stdout:1515 write 0x7651034 thru 0x765c7b2 (0xb77f bytes) 2024-09-14T07:00:03.395 INFO:teuthology.orchestra.run.smithi007.stdout:1516 punch from 0x67902e0 to 0x6791735, (0x1455 bytes) 2024-09-14T07:00:03.395 INFO:teuthology.orchestra.run.smithi007.stdout:1517 write 0x1979cb3 thru 0x197c224 (0x2572 bytes) 2024-09-14T07:00:03.396 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T07:00:03.396 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:03.393+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:03.405 INFO:teuthology.orchestra.run.smithi007.stdout:1518 writesame 0x945d12a thru 0x9460e12 data_size 0x1(0x3ce9 bytes) 2024-09-14T07:00:03.405 INFO:teuthology.orchestra.run.smithi007.stdout:1520 write 0x6bf7f4f thru 0x6c02210 (0xa2c2 bytes) 2024-09-14T07:00:03.413 INFO:teuthology.orchestra.run.smithi007.stdout:1521 read 0x261d7e thru 0x27198a (0xfc0d bytes) 2024-09-14T07:00:03.420 INFO:teuthology.orchestra.run.smithi007.stdout:1522 write 0x30c06cb thru 0x30cc428 (0xbd5e bytes) 2024-09-14T07:00:03.424 INFO:teuthology.orchestra.run.smithi007.stdout:1523 write 0x1db7a81 thru 0x1dbeeaf (0x742f bytes) 2024-09-14T07:00:03.429 INFO:teuthology.orchestra.run.smithi007.stdout:1524 read 0x1359f09 thru 0x1364e4a (0xaf42 bytes) 2024-09-14T07:00:03.429 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:03.429 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:03.429 INFO:teuthology.orchestra.run.smithi007.stdout:1527 read 0x353fd52 thru 0x3541e25 (0x20d4 bytes) 2024-09-14T07:00:03.429 INFO:teuthology.orchestra.run.smithi007.stdout:1529 read 0x749ed thru 0x80f5e (0xc572 bytes) 2024-09-14T07:00:03.435 INFO:teuthology.orchestra.run.smithi007.stdout:1530 read 0x73ebdc1 thru 0x73f6f88 (0xb1c8 bytes) 2024-09-14T07:00:03.435 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:03.435 INFO:teuthology.orchestra.run.smithi007.stdout:1532 write 0x6a27aa9 thru 0x6a2d6e7 (0x5c3f bytes) 2024-09-14T07:00:03.448 INFO:teuthology.orchestra.run.smithi007.stdout:1534 writesame 0xe7c2ccf thru 0xe7c687b data_size 0x1(0x3bad bytes) 2024-09-14T07:00:03.448 INFO:teuthology.orchestra.run.smithi007.stdout:1535 writesame 0x7ad60ac thru 0x7ae5039 data_size 0x9(0xef8e bytes) 2024-09-14T07:00:03.453 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:03.453 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:03.458 INFO:teuthology.orchestra.run.smithi007.stdout:1539 trunc from 0xe7c687c to 0x9d69b0a 2024-09-14T07:00:03.464 INFO:teuthology.orchestra.run.smithi007.stdout:1540 clone 16 order 25 su 262144 sc 7 2024-09-14T07:00:03.495 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:03.494+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:03.504 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:03.504 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:03.504 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:03.512 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:03.596 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:03.595+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:03.655 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:03.655 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:03.655 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:03.655 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:03.655 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:03.663 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:03.696 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:03.696+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:03.798 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:03.797+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:03.814 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:03.814 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:03.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:03.814 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:03.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7619 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25436160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533626+0000", 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.279442053, 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050835010999999999, 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7619 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533626+0000", 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533626+0000", 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533627+0000", 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9499999999999998e-07 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533629+0000", 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.812e-06 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533634+0000", 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9529999999999996e-06 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533639+0000", 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6999999999999999e-06 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.583252+0000", 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049613288999999998 2024-09-14T07:00:03.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.583328+0000", 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5112999999999997e-05 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584461+0000", 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0011338489999999999 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7620 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25550848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533671+0000", 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.279397055, 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050947723, 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7620 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.820 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533671+0000", 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533671+0000", 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533672+0000", 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9800000000000003e-07 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533675+0000", 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0139999999999999e-06 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533686+0000", 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1104000000000001e-05 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533691+0000", 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9189999999999998e-06 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584498+0000", 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050806905999999999 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584547+0000", 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8733999999999998e-05 2024-09-14T07:00:03.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584619+0000", 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2447999999999995e-05 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7621 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25665536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533699+0000", 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.279368975000001, 2024-09-14T07:00:03.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051134163000000003, 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7621 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533699+0000", 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533699+0000", 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533700+0000", 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0400000000000002e-07 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533701+0000", 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4950000000000001e-06 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533705+0000", 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.788e-06 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533709+0000", 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8419999999999998e-06 2024-09-14T07:00:03.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584659+0000", 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050950016000000001 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584747+0000", 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8117999999999999e-05 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584834+0000", 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6500000000000002e-05 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7622 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25780224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533713+0000", 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.279355315, 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051311927, 2024-09-14T07:00:03.829 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7622 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533713+0000", 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533713+0000", 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533713+0000", 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.29e-07 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533716+0000", 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.362e-06 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533719+0000", 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3129999999999999e-06 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533722+0000", 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6259999999999999e-06 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584874+0000", 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051152345000000002 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584960+0000", 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5795000000000001e-05 2024-09-14T07:00:03.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585025+0000", 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5257000000000002e-05 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7623 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25894912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533725+0000", 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.279343348000001, 2024-09-14T07:00:03.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051497312000000003, 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7623 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533725+0000", 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533725+0000", 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533725+0000", 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3099999999999999e-07 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533726+0000", 2024-09-14T07:00:03.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0109999999999999e-06 2024-09-14T07:00:03.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533729+0000", 2024-09-14T07:00:03.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.01e-06 2024-09-14T07:00:03.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533732+0000", 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.57e-06 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585061+0000", 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051328632999999999 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585144+0000", 2024-09-14T07:00:03.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3061999999999999e-05 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585222+0000", 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8794999999999994e-05 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7624 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26009600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.839 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533735+0000", 2024-09-14T07:00:03.840 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.279333565000002, 2024-09-14T07:00:03.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051626949999999998, 2024-09-14T07:00:03.840 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.840 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.840 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.840 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7624 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533735+0000", 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533735+0000", 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533913+0000", 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017811300000000001 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533918+0000", 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6820000000000002e-06 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533932+0000", 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3899e-05 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533939+0000", 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5190000000000003e-06 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585252+0000", 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051313304999999997 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585297+0000", 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4149e-05 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585362+0000", 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5283000000000005e-05 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.844 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7625 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26124288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533952+0000", 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.279116613999999, 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051605167, 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.845 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7625 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533952+0000", 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533952+0000", 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534144+0000", 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019176499999999999 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534147+0000", 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.382e-06 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534156+0000", 2024-09-14T07:00:03.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2289999999999992e-06 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534161+0000", 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7960000000000003e-06 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585402+0000", 2024-09-14T07:00:03.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051240648 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585484+0000", 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1897000000000005e-05 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585557+0000", 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3449999999999996e-05 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.850 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7626 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26238976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.850 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.534171+0000", 2024-09-14T07:00:03.850 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.278897169, 2024-09-14T07:00:03.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051588410000000001, 2024-09-14T07:00:03.850 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.850 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7626 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534171+0000", 2024-09-14T07:00:03.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534171+0000", 2024-09-14T07:00:03.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534297+0000", 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012619899999999999 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534305+0000", 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3329999999999999e-06 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534313+0000", 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4290000000000001e-06 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534320+0000", 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6780000000000003e-06 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585598+0000", 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051277572 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585682+0000", 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4858000000000006e-05 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585760+0000", 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7341000000000004e-05 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7627 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26353664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.534330+0000", 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.278738070999999, 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051630638, 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7627 2024-09-14T07:00:03.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534330+0000", 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534330+0000", 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534576+0000", 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00024544699999999998 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534581+0000", 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7870000000000001e-06 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534589+0000", 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6119999999999999e-06 2024-09-14T07:00:03.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534596+0000", 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6590000000000001e-06 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585801+0000", 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051204972000000001 2024-09-14T07:00:03.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585883+0000", 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2494e-05 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585961+0000", 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7667e-05 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.860 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7628 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26468352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.534609+0000", 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.278459946000002, 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051474580999999998, 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.861 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7628 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534609+0000", 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534609+0000", 2024-09-14T07:00:03.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538869+0000", 2024-09-14T07:00:03.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042609450000000004 2024-09-14T07:00:03.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538874+0000", 2024-09-14T07:00:03.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5050000000000001e-06 2024-09-14T07:00:03.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538884+0000", 2024-09-14T07:00:03.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0242e-05 2024-09-14T07:00:03.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538891+0000", 2024-09-14T07:00:03.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2119999999999997e-06 2024-09-14T07:00:03.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585991+0000", 2024-09-14T07:00:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.047099540000000002 2024-09-14T07:00:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.586038+0000", 2024-09-14T07:00:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6971000000000001e-05 2024-09-14T07:00:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.586083+0000", 2024-09-14T07:00:03.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5166000000000001e-05 2024-09-14T07:00:03.867 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.867 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.867 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.867 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7644 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28319744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.867 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539166+0000", 2024-09-14T07:00:03.868 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.273902042, 2024-09-14T07:00:03.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049804237000000001, 2024-09-14T07:00:03.868 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.868 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.868 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.868 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.868 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.868 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.869 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7644 2024-09-14T07:00:03.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.869 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539166+0000", 2024-09-14T07:00:03.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539166+0000", 2024-09-14T07:00:03.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539167+0000", 2024-09-14T07:00:03.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:00:03.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539168+0000", 2024-09-14T07:00:03.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7100000000000011e-07 2024-09-14T07:00:03.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539172+0000", 2024-09-14T07:00:03.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0980000000000004e-06 2024-09-14T07:00:03.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539175+0000", 2024-09-14T07:00:03.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0319999999999999e-06 2024-09-14T07:00:03.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.588864+0000", 2024-09-14T07:00:03.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049688831000000003 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.588928+0000", 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4193e-05 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.588971+0000", 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2910000000000001e-05 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7645 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28434432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539179+0000", 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.273889039, 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04998959, 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7645 2024-09-14T07:00:03.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539179+0000", 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539179+0000", 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539180+0000", 2024-09-14T07:00:03.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:00:03.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539182+0000", 2024-09-14T07:00:03.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4229999999999999e-06 2024-09-14T07:00:03.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539189+0000", 2024-09-14T07:00:03.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6270000000000002e-06 2024-09-14T07:00:03.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539194+0000", 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4970000000000001e-06 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589004+0000", 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049809785000000002 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589091+0000", 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7348999999999993e-05 2024-09-14T07:00:03.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589169+0000", 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7655000000000001e-05 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7646 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28549120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539202+0000", 2024-09-14T07:00:03.880 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.273866752, 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050152584, 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7646 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539202+0000", 2024-09-14T07:00:03.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539202+0000", 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539202+0000", 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6199999999999999e-07 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539203+0000", 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.218e-06 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539207+0000", 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2540000000000003e-06 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539210+0000", 2024-09-14T07:00:03.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.757e-06 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589199+0000", 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049988800999999999 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589275+0000", 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6174000000000006e-05 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589354+0000", 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9117999999999997e-05 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7647 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28663808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539216+0000", 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.273852036000001, 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050330617000000001, 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.886 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7647 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539216+0000", 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539216+0000", 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539217+0000", 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539218+0000", 2024-09-14T07:00:03.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6589999999999999e-06 2024-09-14T07:00:03.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539225+0000", 2024-09-14T07:00:03.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2859999999999996e-06 2024-09-14T07:00:03.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539229+0000", 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589391+0000", 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0501623 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589472+0000", 2024-09-14T07:00:03.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0752000000000002e-05 2024-09-14T07:00:03.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589547+0000", 2024-09-14T07:00:03.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5018999999999994e-05 2024-09-14T07:00:03.891 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.891 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.891 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.892 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7648 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28778496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.892 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539236+0000", 2024-09-14T07:00:03.892 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.273832766999998, 2024-09-14T07:00:03.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050463816000000002, 2024-09-14T07:00:03.892 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.892 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.892 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.893 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.893 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.893 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.893 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7648 2024-09-14T07:00:03.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.893 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539236+0000", 2024-09-14T07:00:03.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539236+0000", 2024-09-14T07:00:03.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539236+0000", 2024-09-14T07:00:03.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:00:03.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539237+0000", 2024-09-14T07:00:03.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.263e-06 2024-09-14T07:00:03.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539241+0000", 2024-09-14T07:00:03.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3960000000000002e-06 2024-09-14T07:00:03.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539243+0000", 2024-09-14T07:00:03.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:00:03.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589586+0000", 2024-09-14T07:00:03.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050343176000000003 2024-09-14T07:00:03.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589656+0000", 2024-09-14T07:00:03.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9530999999999995e-05 2024-09-14T07:00:03.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589700+0000", 2024-09-14T07:00:03.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3511999999999999e-05 2024-09-14T07:00:03.898 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.899 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.899 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.899 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7649 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28893184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.899 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539247+0000", 2024-09-14T07:00:03.899 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.273821896000001, 2024-09-14T07:00:03.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050647407999999998, 2024-09-14T07:00:03.900 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.900 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.900 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.900 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.900 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.900 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.900 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7649 2024-09-14T07:00:03.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.900 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539247+0000", 2024-09-14T07:00:03.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539247+0000", 2024-09-14T07:00:03.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539247+0000", 2024-09-14T07:00:03.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:03.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539248+0000", 2024-09-14T07:00:03.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0279999999999999e-06 2024-09-14T07:00:03.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539251+0000", 2024-09-14T07:00:03.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9019999999999999e-06 2024-09-14T07:00:03.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539253+0000", 2024-09-14T07:00:03.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2989999999999999e-06 2024-09-14T07:00:03.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589733+0000", 2024-09-14T07:00:03.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050480216000000001 2024-09-14T07:00:03.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589817+0000", 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4159000000000005e-05 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589894+0000", 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6595999999999996e-05 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7650 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 29007872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539256+0000", 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.273812691, 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050806128999999998, 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7650 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.906 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539256+0000", 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539256+0000", 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539256+0000", 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9999999999999999e-07 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539261+0000", 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7820000000000001e-06 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539265+0000", 2024-09-14T07:00:03.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1219999999999997e-06 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539267+0000", 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5689999999999999e-06 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589925+0000", 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050657695000000003 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589979+0000", 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4088999999999997e-05 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590062+0000", 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2671999999999997e-05 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7651 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 29122560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539271+0000", 2024-09-14T07:00:03.910 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.273797647999999, 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050987034000000001, 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7651 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539271+0000", 2024-09-14T07:00:03.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539271+0000", 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539271+0000", 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539272+0000", 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4099999999999997e-07 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539275+0000", 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0699999999999998e-06 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539278+0000", 2024-09-14T07:00:03.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5179999999999999e-06 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590100+0000", 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050822289 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590181+0000", 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1269999999999999e-05 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590258+0000", 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6737999999999996e-05 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7652 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 29237248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539280+0000", 2024-09-14T07:00:03.915 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.273788120999999, 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051152192999999999, 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7652 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539280+0000", 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539280+0000", 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543135+0000", 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0038546919999999998 2024-09-14T07:00:03.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543140+0000", 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543151+0000", 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1314e-05 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543156+0000", 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9590000000000003e-06 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590297+0000", 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.047140973000000003 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590381+0000", 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4006999999999997e-05 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590433+0000", 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1456999999999999e-05 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7654 3.2 3:41572fcc:::rbd_object_map.114f54fbf0ed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:03.920 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.543182+0000", 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.269886510999999, 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050091802999999997, 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7654 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:03.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543182+0000", 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543182+0000", 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543182+0000", 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6600000000000002e-07 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543184+0000", 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.063e-06 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543188+0000", 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8680000000000001e-06 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543191+0000", 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.881e-06 2024-09-14T07:00:03.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590653+0000", 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.047461934999999997 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590706+0000", 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3340000000000001e-05 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.591041+0000", 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00033495900000000002 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593167+0000", 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021259389999999999 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593183+0000", 2024-09-14T07:00:03.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5515e-05 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593233+0000", 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9777000000000001e-05 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593274+0000", 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1159999999999999e-05 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.926 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:03.927 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:03.927 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:03.927 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:03.928 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:03.928+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:04.030 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:04.029+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:04.031 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:04.031 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:04.031 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:04.031 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:04.039 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:04.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:04.142 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:04.142 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:04.142 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:04.142 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:04.142 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:04.151 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:04.154 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:04.154+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:04.232 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:04.231+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:04.244 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:04.245 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:04.257 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:04.257 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:04.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:04.257 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:04.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15710 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8118272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165219+0000", 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.090754215, 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043978966000000001, 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.258 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15710 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165219+0000", 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165219+0000", 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165347+0000", 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000128081 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165350+0000", 2024-09-14T07:00:04.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1870000000000001e-06 2024-09-14T07:00:04.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165361+0000", 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0380000000000001e-05 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165367+0000", 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2559999999999999e-06 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209056+0000", 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043688762999999999 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209131+0000", 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5637999999999996e-05 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209198+0000", 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6661000000000004e-05 2024-09-14T07:00:04.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15711 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8232960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165377+0000", 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.090596425000001, 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043969613999999997, 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.264 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15711 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165377+0000", 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165377+0000", 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165413+0000", 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6238000000000001e-05 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165417+0000", 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165424+0000", 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5429999999999996e-06 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165430+0000", 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8880000000000002e-06 2024-09-14T07:00:04.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209227+0000", 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043797520999999999 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209302+0000", 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4234999999999997e-05 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209347+0000", 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4851000000000001e-05 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.090528456000001, 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:00:04.269 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:04.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:00:04.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:00:04.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:00:04.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:00:04.274 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.274 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.274 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87100 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1562624~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.470949+0000", 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.785024202, 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043926004999999997, 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.275 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87100 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.470949+0000", 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.470949+0000", 2024-09-14T07:00:04.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471088+0000", 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013925100000000001 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471092+0000", 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7330000000000001e-06 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471101+0000", 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9800000000000004e-06 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471107+0000", 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0569999999999998e-06 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.514765+0000", 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043658107000000002 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.514816+0000", 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0618999999999998e-05 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.514875+0000", 2024-09-14T07:00:04.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9258000000000001e-05 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87101 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1644544~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.471121+0000", 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7848525790000001, 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043957714000000002, 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.280 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87101 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471121+0000", 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471121+0000", 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471251+0000", 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012987 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471258+0000", 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2099999999999996e-06 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471273+0000", 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4785e-05 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471280+0000", 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0659999999999998e-06 2024-09-14T07:00:04.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.514912+0000", 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043631793000000002 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.514987+0000", 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5338000000000003e-05 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515079+0000", 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1651999999999996e-05 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87102 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1654784~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.471293+0000", 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7846808109999999, 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043918594999999998, 2024-09-14T07:00:04.285 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87102 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471293+0000", 2024-09-14T07:00:04.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471293+0000", 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471439+0000", 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014643699999999999 2024-09-14T07:00:04.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471443+0000", 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5810000000000001e-06 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471451+0000", 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3280000000000006e-06 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471462+0000", 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1318000000000001e-05 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515107+0000", 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043644863999999998 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515152+0000", 2024-09-14T07:00:04.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4481999999999999e-05 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515211+0000", 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9585e-05 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87103 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1665024~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.290 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.471474+0000", 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7844991079999999, 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043941998000000003, 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87103 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.291 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471474+0000", 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471474+0000", 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471618+0000", 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014352700000000001 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471621+0000", 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6430000000000001e-06 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471629+0000", 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0420000000000003e-06 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471636+0000", 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2790000000000004e-06 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515246+0000", 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043610271999999999 2024-09-14T07:00:04.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515325+0000", 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.907e-05 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515416+0000", 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1164999999999999e-05 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87104 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1736704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.471646+0000", 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7843275379999999, 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043903975999999997, 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.296 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87104 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471646+0000", 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471646+0000", 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471780+0000", 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000133774 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471783+0000", 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6009999999999999e-06 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471793+0000", 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6490000000000003e-06 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471805+0000", 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1790999999999999e-05 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515445+0000", 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043640237999999998 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515489+0000", 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4404999999999999e-05 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515550+0000", 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0517999999999999e-05 2024-09-14T07:00:04.300 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87105 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1746944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.471825+0000", 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.78414826, 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043932475999999998, 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.301 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87105 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471825+0000", 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471825+0000", 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471932+0000", 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000106699 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471935+0000", 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6650000000000001e-06 2024-09-14T07:00:04.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471949+0000", 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3844e-05 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471956+0000", 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7299999999999999e-06 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515585+0000", 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043628843 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515664+0000", 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8724e-05 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515758+0000", 2024-09-14T07:00:04.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3970999999999999e-05 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87106 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1757184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.471967+0000", 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7840068179999999, 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043927150999999998, 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.306 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87106 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471967+0000", 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.471967+0000", 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472119+0000", 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015226199999999999 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472122+0000", 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.641e-06 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472131+0000", 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6649999999999992e-06 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472137+0000", 2024-09-14T07:00:04.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9939999999999997e-06 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515786+0000", 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043648691000000003 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515834+0000", 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8322999999999998e-05 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515894+0000", 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9574999999999998e-05 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87107 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1767424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.472147+0000", 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.783826664, 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043954761000000002, 2024-09-14T07:00:04.311 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87107 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472147+0000", 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472147+0000", 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472261+0000", 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114233 2024-09-14T07:00:04.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472264+0000", 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4879999999999999e-06 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472279+0000", 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4285e-05 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472285+0000", 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2060000000000004e-06 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.515930+0000", 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043644609000000001 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516009+0000", 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9164000000000003e-05 2024-09-14T07:00:04.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516101+0000", 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2775999999999994e-05 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87108 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1839104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.472296+0000", 2024-09-14T07:00:04.316 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7836776599999999, 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043939898999999998, 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87108 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472296+0000", 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472296+0000", 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472622+0000", 2024-09-14T07:00:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032658399999999999 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472626+0000", 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.602e-06 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472641+0000", 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4616e-05 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472647+0000", 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.494e-06 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516129+0000", 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043481995000000002 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516175+0000", 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5698000000000002e-05 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516236+0000", 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0909999999999998e-05 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87110 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1859584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.321 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.472711+0000", 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.783262361, 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043861381999999997, 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87110 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472711+0000", 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472711+0000", 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472712+0000", 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0399999999999996e-07 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472714+0000", 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5909999999999998e-06 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472721+0000", 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9870000000000002e-06 2024-09-14T07:00:04.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472727+0000", 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4700000000000001e-06 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516467+0000", 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043740791000000001 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516513+0000", 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5067000000000001e-05 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516572+0000", 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9871999999999999e-05 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87111 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1941504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.472740+0000", 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.783233555, 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044040330000000003, 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87111 2024-09-14T07:00:04.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472740+0000", 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472740+0000", 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472937+0000", 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000197324 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472941+0000", 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6260000000000001e-06 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472951+0000", 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0210000000000001e-05 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472957+0000", 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9920000000000004e-06 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516608+0000", 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043651185000000002 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516686+0000", 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7954000000000006e-05 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516780+0000", 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4038999999999999e-05 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87112 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1951744~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.472970+0000", 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7830032149999999, 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043943742000000001, 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.332 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87112 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472970+0000", 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.472970+0000", 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473087+0000", 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011689 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473090+0000", 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.439e-06 2024-09-14T07:00:04.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473105+0000", 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4242e-05 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473111+0000", 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9510000000000003e-06 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516811+0000", 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043700441999999999 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516855+0000", 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4257e-05 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516914+0000", 2024-09-14T07:00:04.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8520999999999998e-05 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87113 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1961984~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.473120+0000", 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7828531439999999, 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043995932000000001, 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.337 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87113 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473120+0000", 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473120+0000", 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473225+0000", 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010429000000000001 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473228+0000", 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2150000000000001e-06 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473235+0000", 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6340000000000001e-06 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473241+0000", 2024-09-14T07:00:04.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4040000000000003e-06 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.516950+0000", 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043709421999999998 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517025+0000", 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4658000000000002e-05 2024-09-14T07:00:04.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517116+0000", 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.1309000000000003e-05 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87114 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2033664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.473250+0000", 2024-09-14T07:00:04.342 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7827230279999999, 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043995099000000003, 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87114 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473250+0000", 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473250+0000", 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473384+0000", 2024-09-14T07:00:04.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000133311 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473387+0000", 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4230000000000001e-06 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473395+0000", 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6120000000000001e-06 2024-09-14T07:00:04.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473400+0000", 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517143+0000", 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043742953000000001 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517189+0000", 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5516000000000003e-05 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517245+0000", 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6524000000000003e-05 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.347 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87115 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2043904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.473410+0000", 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.782563782, 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044041309000000001, 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87115 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.348 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473410+0000", 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473410+0000", 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473750+0000", 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00033991799999999998 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473753+0000", 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.895e-06 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473760+0000", 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4280000000000001e-06 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473764+0000", 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3810000000000001e-06 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517282+0000", 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043517732000000003 2024-09-14T07:00:04.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517357+0000", 2024-09-14T07:00:04.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4846999999999997e-05 2024-09-14T07:00:04.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517451+0000", 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4108000000000001e-05 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87117 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2136064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.473958+0000", 2024-09-14T07:00:04.353 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.782015251, 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043829371999999998, 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87117 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473958+0000", 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.473958+0000", 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474073+0000", 2024-09-14T07:00:04.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114537 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474076+0000", 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7479999999999999e-06 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474082+0000", 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9989999999999997e-06 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474088+0000", 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1739999999999999e-06 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517614+0000", 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043526408000000003 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517693+0000", 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9076e-05 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517788+0000", 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4430000000000002e-05 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:87118 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2146304~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52)", 2024-09-14T07:00:04.358 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:02.474096+0000", 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.7818771170000001, 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043830322999999997, 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 87118 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.359 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474096+0000", 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474096+0000", 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:04.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474355+0000", 2024-09-14T07:00:04.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025833900000000001 2024-09-14T07:00:04.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474359+0000", 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9979999999999997e-06 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474370+0000", 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1552000000000001e-05 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.474376+0000", 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1680000000000001e-06 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517822+0000", 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043445442000000001 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:04.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517867+0000", 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5241999999999998e-05 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:02.517927+0000", 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9582e-05 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:04.364 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:04.365 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:04.366 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:04.366 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:04.367 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:04.367 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:04.368 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:04.365+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:04.368 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:04.368 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:04.369 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:04.369 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:04.369 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:04.369 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:04.370 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:04.370 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:04.370 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:04.467 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:04.466+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:04.509 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone15 from 0x9d69b0a (overlap 0xc6e86c) to 0x4f902b 2024-09-14T07:00:04.513 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:04.513 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:04.526 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:04.572 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:04.571+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:04.640 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:04.640 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:04.648 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:04.690 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:04.689+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:04.769 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:04.768+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:04.816 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:04.816 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:04.825 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:04.909 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:04.908+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:04.950 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-14T07:00:04.971 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:04.970+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:04.983 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:04.983 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:04.993 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:05.072 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:05.072+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:05.143 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:05.143 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:05.152 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:05.173 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:05.173+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:05.264 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:05.264 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:05.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:05.274 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:05.273+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:05.327 INFO:teuthology.orchestra.run.smithi007.stdout:1541 read 0x498f0de thru 0x499ac6d (0xbb90 bytes) 2024-09-14T07:00:05.342 INFO:teuthology.orchestra.run.smithi007.stdout:1542 write 0xdcc86a3 thru 0xdcc9025 (0x983 bytes) 2024-09-14T07:00:05.375 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:05.375+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:05.388 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:05.388 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:05.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:05.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:00:05.415 INFO:teuthology.orchestra.run.smithi007.stdout:1543 punch from 0x8e3af99 to 0x8e3c82e, (0x1895 bytes) 2024-09-14T07:00:05.415 INFO:teuthology.orchestra.run.smithi007.stdout:1544 punch from 0x475b8f7 to 0x475f85c, (0x3f65 bytes) 2024-09-14T07:00:05.415 INFO:teuthology.orchestra.run.smithi007.stdout:1545 writesame 0x8537958 thru 0x854456f data_size 0x54(0xcc18 bytes) 2024-09-14T07:00:05.464 INFO:teuthology.orchestra.run.smithi007.stdout:1546 write 0x4bf4768 thru 0x4c02920 (0xe1b9 bytes) 2024-09-14T07:00:05.476 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:05.476+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:05.521 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:05.521 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:05.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:05.572 INFO:teuthology.orchestra.run.smithi007.stdout:1547 writesame 0xe57311a thru 0xe57f1cd data_size 0x6(0xc0b4 bytes) 2024-09-14T07:00:05.573 INFO:teuthology.orchestra.run.smithi007.stdout:1548 punch from 0x157965e to 0x157e673, (0x5015 bytes) 2024-09-14T07:00:05.573 INFO:teuthology.orchestra.run.smithi007.stdout:1549 writesame 0x97b78ee thru 0x97bbe8f data_size 0x3(0x45a2 bytes) 2024-09-14T07:00:05.577 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:05.576+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:1551 read 0xbc5a323 thru 0xbc66c1b (0xc8f9 bytes) 2024-09-14T07:00:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:1552 read 0xb477e82 thru 0xb47a508 (0x2687 bytes) 2024-09-14T07:00:05.630 INFO:teuthology.orchestra.run.smithi007.stdout:1553 write 0x9070700 thru 0x9075c8a (0x558b bytes) 2024-09-14T07:00:05.632 INFO:teuthology.orchestra.run.smithi007.stdout:1554 read 0xc16bb21 thru 0xc173670 (0x7b50 bytes) 2024-09-14T07:00:05.632 INFO:teuthology.orchestra.run.smithi007.stdout:1556 punch from 0x1c10a87 to 0x1c1272f, (0x1ca8 bytes) 2024-09-14T07:00:05.632 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.632 INFO:teuthology.orchestra.run.smithi007.stdout:1559 punch from 0x6828919 to 0x683610a, (0xd7f1 bytes) 2024-09-14T07:00:05.632 INFO:teuthology.orchestra.run.smithi007.stdout:1561 write 0x3c7c01d thru 0x3c81644 (0x5628 bytes) 2024-09-14T07:00:05.643 INFO:teuthology.orchestra.run.smithi007.stdout:1563 trunc from 0xe57f1ce to 0x60ec19c 2024-09-14T07:00:05.661 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:05.661 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:05.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:05.704 INFO:teuthology.orchestra.run.smithi007.stdout:1564 punch from 0x5ff1e50 to 0x5ffc3a3, (0xa553 bytes) 2024-09-14T07:00:05.704 INFO:teuthology.orchestra.run.smithi007.stdout:1565 write 0x48d0c67 thru 0x48d105f (0x3f9 bytes) 2024-09-14T07:00:05.705 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:05.703+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:05.706 INFO:teuthology.orchestra.run.smithi007.stdout:1566 read 0x86d6cb thru 0x87b197 (0xdacd bytes) 2024-09-14T07:00:05.715 INFO:teuthology.orchestra.run.smithi007.stdout:1567 write 0xa0a4113 thru 0xa0af223 (0xb111 bytes) 2024-09-14T07:00:05.717 INFO:teuthology.orchestra.run.smithi007.stdout:1568 read 0x10f1899 thru 0x10f79ae (0x6116 bytes) 2024-09-14T07:00:05.718 INFO:teuthology.orchestra.run.smithi007.stdout:1569 trunc from 0xa0af224 to 0xad4e41b 2024-09-14T07:00:05.724 INFO:teuthology.orchestra.run.smithi007.stdout:1570 write 0x336c060 thru 0x33719f9 (0x599a bytes) 2024-09-14T07:00:05.726 INFO:teuthology.orchestra.run.smithi007.stdout:1572 read 0x1623f04 thru 0x162f3c9 (0xb4c6 bytes) 2024-09-14T07:00:05.726 INFO:teuthology.orchestra.run.smithi007.stdout:1574 punch from 0x39f53dc to 0x39f5ee4, (0xb08 bytes) 2024-09-14T07:00:05.726 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.727 INFO:teuthology.orchestra.run.smithi007.stdout:1576 writesame 0x89df1ae thru 0x89ec0c1 data_size 0x4(0xcf14 bytes) 2024-09-14T07:00:05.732 INFO:teuthology.orchestra.run.smithi007.stdout:1577 writesame 0x5032576 thru 0x5040b31 data_size 0xd(0xe5bc bytes) 2024-09-14T07:00:05.734 INFO:teuthology.orchestra.run.smithi007.stdout:1578 writesame 0x59632b8 thru 0x5970454 data_size 0x1f(0xd19d bytes) 2024-09-14T07:00:05.737 INFO:teuthology.orchestra.run.smithi007.stdout:1580 write 0x36e904b thru 0x36ea2ea (0x12a0 bytes) 2024-09-14T07:00:05.738 INFO:teuthology.orchestra.run.smithi007.stdout:1581 write 0x1a5b610 thru 0x1a685a6 (0xcf97 bytes) 2024-09-14T07:00:05.740 INFO:teuthology.orchestra.run.smithi007.stdout:1583 read 0x9906535 thru 0x99084c1 (0x1f8d bytes) 2024-09-14T07:00:05.741 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.741 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.741 INFO:teuthology.orchestra.run.smithi007.stdout:1586 punch from 0x137738c to 0x137be9c, (0x4b10 bytes) 2024-09-14T07:00:05.741 INFO:teuthology.orchestra.run.smithi007.stdout:1587 write 0x55eacbf thru 0x55f8732 (0xda74 bytes) 2024-09-14T07:00:05.742 INFO:teuthology.orchestra.run.smithi007.stdout:1591 writesame 0x3253a04 thru 0x3261d37 data_size 0x83(0xe334 bytes) 2024-09-14T07:00:05.755 INFO:teuthology.orchestra.run.smithi007.stdout:1592 writesame 0xd90d08e thru 0xd90f85b data_size 0x5(0x27ce bytes) 2024-09-14T07:00:05.756 INFO:teuthology.orchestra.run.smithi007.stdout:1594 read 0xd462979 thru 0xd471a70 (0xf0f8 bytes) 2024-09-14T07:00:05.756 INFO:teuthology.orchestra.run.smithi007.stdout:1595 writesame 0x8302d9d thru 0x830313a data_size 0x1(0x39e bytes) 2024-09-14T07:00:05.758 INFO:teuthology.orchestra.run.smithi007.stdout:1597 punch from 0xbbd81e1 to 0xbbe2467, (0xa286 bytes) 2024-09-14T07:00:05.759 INFO:teuthology.orchestra.run.smithi007.stdout:1598 write 0x74573ed thru 0x7460e3e (0x9a52 bytes) 2024-09-14T07:00:05.760 INFO:teuthology.orchestra.run.smithi007.stdout:1600 writesame 0xc5ace78 thru 0xc5b285a data_size 0x1(0x59e3 bytes) 2024-09-14T07:00:05.771 INFO:teuthology.orchestra.run.smithi007.stdout:1601 write 0xb68098e thru 0xb68395a (0x2fcd bytes) 2024-09-14T07:00:05.773 INFO:teuthology.orchestra.run.smithi007.stdout:1602 read 0x8afc34f thru 0x8afdfa9 (0x1c5b bytes) 2024-09-14T07:00:05.773 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.773 INFO:teuthology.orchestra.run.smithi007.stdout:1604 write 0x5cbb29e thru 0x5cbf8fd (0x4660 bytes) 2024-09-14T07:00:05.775 INFO:teuthology.orchestra.run.smithi007.stdout:1606 write 0xcca689f thru 0xccb36ea (0xce4c bytes) 2024-09-14T07:00:05.776 INFO:teuthology.orchestra.run.smithi007.stdout:1607 read 0x425ec6d thru 0x42699c3 (0xad57 bytes) 2024-09-14T07:00:05.777 INFO:teuthology.orchestra.run.smithi007.stdout:1608 write 0xa21d2ac thru 0xa22cac0 (0xf815 bytes) 2024-09-14T07:00:05.778 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:05.778+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:05.793 INFO:teuthology.orchestra.run.smithi007.stdout:1609 trunc from 0xd90f85c to 0x1a2e99a 2024-09-14T07:00:05.799 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.826 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:05.827 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:05.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:05.844 INFO:teuthology.orchestra.run.smithi007.stdout:1611 writesame 0xa7c8294 thru 0xa7d0a7d data_size 0x9(0x87ea bytes) 2024-09-14T07:00:05.849 INFO:teuthology.orchestra.run.smithi007.stdout:1612 trunc from 0xa7d0a7e to 0x7a709a1 2024-09-14T07:00:05.856 INFO:teuthology.orchestra.run.smithi007.stdout:1613 read 0x16396bb thru 0x164292a (0x9270 bytes) 2024-09-14T07:00:05.857 INFO:teuthology.orchestra.run.smithi007.stdout:1615 read 0x7289c44 thru 0x7289f41 (0x2fe bytes) 2024-09-14T07:00:05.869 INFO:teuthology.orchestra.run.smithi007.stdout:1617 write 0xddd19f7 thru 0xdde0845 (0xee4f bytes) 2024-09-14T07:00:05.871 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.871 INFO:teuthology.orchestra.run.smithi007.stdout:1619 punch from 0x7c75a7d to 0x7c7bf9a, (0x651d bytes) 2024-09-14T07:00:05.871 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.879 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:05.879+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:05.886 INFO:teuthology.orchestra.run.smithi007.stdout:1623 trunc from 0xdde0846 to 0x595924 2024-09-14T07:00:05.893 INFO:teuthology.orchestra.run.smithi007.stdout:1624 punch from 0x1f8fb to 0x26afb, (0x7200 bytes) 2024-09-14T07:00:05.894 INFO:teuthology.orchestra.run.smithi007.stdout:1626 punch from 0x7e936 to 0x8427e, (0x5948 bytes) 2024-09-14T07:00:05.894 INFO:teuthology.orchestra.run.smithi007.stdout:1627 punch from 0x148882 to 0x14d2fc, (0x4a7a bytes) 2024-09-14T07:00:05.903 INFO:teuthology.orchestra.run.smithi007.stdout:1628 writesame 0x3779daf thru 0x3783bc6 data_size 0x4(0x9e18 bytes) 2024-09-14T07:00:05.914 INFO:teuthology.orchestra.run.smithi007.stdout:1629 trunc from 0x3783bc7 to 0xc232b8b 2024-09-14T07:00:05.919 INFO:teuthology.orchestra.run.smithi007.stdout:1630 read 0x4d5aae8 thru 0x4d5e1e0 (0x36f9 bytes) 2024-09-14T07:00:05.920 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.920 INFO:teuthology.orchestra.run.smithi007.stdout:1633 read 0x11fee4a thru 0x120767d (0x8834 bytes) 2024-09-14T07:00:05.920 INFO:teuthology.orchestra.run.smithi007.stdout:1634 read 0x8015bd1 thru 0x8019deb (0x421b bytes) 2024-09-14T07:00:05.933 INFO:teuthology.orchestra.run.smithi007.stdout:1635 trunc from 0xc232b8b to 0x11f3636 2024-09-14T07:00:05.940 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.955 INFO:teuthology.orchestra.run.smithi007.stdout:1638 write 0x935d43f thru 0x935f1a8 (0x1d6a bytes) 2024-09-14T07:00:05.956 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.956 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.961 INFO:teuthology.orchestra.run.smithi007.stdout:1644 writesame 0x96a748d thru 0x96b6398 data_size 0x2(0xef0c bytes) 2024-09-14T07:00:05.962 INFO:teuthology.orchestra.run.smithi007.stdout:1645 read 0x1f08b34 thru 0x1f0d6ab (0x4b78 bytes) 2024-09-14T07:00:05.971 INFO:teuthology.orchestra.run.smithi007.stdout:1647 write 0xc6a1771 thru 0xc6a343c (0x1ccc bytes) 2024-09-14T07:00:05.971 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:05.972 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:05.973 INFO:teuthology.orchestra.run.smithi007.stdout:1650 writesame 0x28bfd2f thru 0x28c7a40 data_size 0x7(0x7d12 bytes) 2024-09-14T07:00:05.975 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:05.975 INFO:teuthology.orchestra.run.smithi007.stdout:1654 write 0x8f01cf6 thru 0x8f04e83 (0x318e bytes) 2024-09-14T07:00:05.977 INFO:teuthology.orchestra.run.smithi007.stdout:1655 punch from 0x3b128b to 0x3bf7ed, (0xe562 bytes) 2024-09-14T07:00:05.977 INFO:teuthology.orchestra.run.smithi007.stdout:1656 write 0x829b64b thru 0x82a8157 (0xcb0d bytes) 2024-09-14T07:00:05.979 INFO:teuthology.orchestra.run.smithi007.stdout:1658 write 0x3966b3c thru 0x3975e91 (0xf356 bytes) 2024-09-14T07:00:05.980 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:05.979+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:05.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:05.989 INFO:teuthology.orchestra.run.smithi007.stdout:1659 trunc from 0xc6a343d to 0x4ee8881 2024-09-14T07:00:06.008 INFO:teuthology.orchestra.run.smithi007.stdout:1660 write 0xa52ae5f thru 0xa534a43 (0x9be5 bytes) 2024-09-14T07:00:06.010 INFO:teuthology.orchestra.run.smithi007.stdout:1662 write 0x70a679a thru 0x70b0262 (0x9ac9 bytes) 2024-09-14T07:00:06.011 INFO:teuthology.orchestra.run.smithi007.stdout:1663 write 0x418b261 thru 0x4197c5e (0xc9fe bytes) 2024-09-14T07:00:06.020 INFO:teuthology.orchestra.run.smithi007.stdout:1666 trunc from 0xa534a44 to 0x4933058 2024-09-14T07:00:06.026 INFO:teuthology.orchestra.run.smithi007.stdout:1668 read 0xa1ad15 thru 0xa23020 (0x830c bytes) 2024-09-14T07:00:06.039 INFO:teuthology.orchestra.run.smithi007.stdout:1669 writesame 0xabd9ca1 thru 0xabe5a4f data_size 0x7(0xbdaf bytes) 2024-09-14T07:00:06.039 INFO:teuthology.orchestra.run.smithi007.stdout:1670 read 0x46a361c thru 0x46abb05 (0x84ea bytes) 2024-09-14T07:00:06.040 INFO:teuthology.orchestra.run.smithi007.stdout:1671 write 0x3995026 thru 0x399520a (0x1e5 bytes) 2024-09-14T07:00:06.042 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.042 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.046 INFO:teuthology.orchestra.run.smithi007.stdout:1674 trunc from 0xabe5a50 to 0xe3c922c 2024-09-14T07:00:06.050 INFO:teuthology.orchestra.run.smithi007.stdout:1675 writesame 0xda47810 thru 0xda54d73 data_size 0xe(0xd564 bytes) 2024-09-14T07:00:06.066 INFO:teuthology.orchestra.run.smithi007.stdout:1678 trunc from 0xe3c922c to 0x28ebe73 2024-09-14T07:00:06.081 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:06.081+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:06.084 INFO:teuthology.orchestra.run.smithi007.stdout:1679 write 0x7c2569c thru 0x7c25f26 (0x88b bytes) 2024-09-14T07:00:06.085 INFO:teuthology.orchestra.run.smithi007.stdout:1681 write 0x75ad889 thru 0x75b029a (0x2a12 bytes) 2024-09-14T07:00:06.087 INFO:teuthology.orchestra.run.smithi007.stdout:1683 punch from 0x669a045 to 0x66a0bcc, (0x6b87 bytes) 2024-09-14T07:00:06.087 INFO:teuthology.orchestra.run.smithi007.stdout:1684 read 0x11fc151 thru 0x1208c99 (0xcb49 bytes) 2024-09-14T07:00:06.096 INFO:teuthology.orchestra.run.smithi007.stdout:1686 write 0xaa91262 thru 0xaa92c19 (0x19b8 bytes) 2024-09-14T07:00:06.097 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.105 INFO:teuthology.orchestra.run.smithi007.stdout:1690 write 0xc5c97bc thru 0xc5d4de4 (0xb629 bytes) 2024-09-14T07:00:06.106 INFO:teuthology.orchestra.run.smithi007.stdout:1691 read 0x4a6082f thru 0x4a60d6c (0x53e bytes) 2024-09-14T07:00:06.107 INFO:teuthology.orchestra.run.smithi007.stdout:1692 read 0x388a289 thru 0x3899e90 (0xfc08 bytes) 2024-09-14T07:00:06.107 INFO:teuthology.orchestra.run.smithi007.stdout:1693 writesame 0x1c45f6a thru 0x1c47a3d data_size 0x11(0x1ad4 bytes) 2024-09-14T07:00:06.115 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:06.115 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:06.117 INFO:teuthology.orchestra.run.smithi007.stdout:1694 writesame 0xe34be51 thru 0xe34f579 data_size 0x9(0x3729 bytes) 2024-09-14T07:00:06.125 INFO:teuthology.orchestra.run.smithi007.stdout:1695 trunc from 0xe34f57a to 0x72b9b9b 2024-09-14T07:00:06.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:06.132 INFO:teuthology.orchestra.run.smithi007.stdout:1696 writesame 0x5100257 thru 0x510dc60 data_size 0x3f(0xda0a bytes) 2024-09-14T07:00:06.134 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.143 INFO:teuthology.orchestra.run.smithi007.stdout:1701 write 0xa29c89b thru 0xa2ac5f7 (0xfd5d bytes) 2024-09-14T07:00:06.145 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.145 INFO:teuthology.orchestra.run.smithi007.stdout:1705 writesame 0x31a8adb thru 0x31b8246 data_size 0xa(0xf76c bytes) 2024-09-14T07:00:06.149 INFO:teuthology.orchestra.run.smithi007.stdout:1706 write 0x5d597db thru 0x5d68dfb (0xf621 bytes) 2024-09-14T07:00:06.150 INFO:teuthology.orchestra.run.smithi007.stdout:1707 write 0x4ff60f thru 0x5005c7 (0xfb9 bytes) 2024-09-14T07:00:06.153 INFO:teuthology.orchestra.run.smithi007.stdout:1708 writesame 0x51b1606 thru 0x51b48cf data_size 0x16(0x32ca bytes) 2024-09-14T07:00:06.157 INFO:teuthology.orchestra.run.smithi007.stdout:1709 trunc from 0xa2ac5f8 to 0x8324e47 2024-09-14T07:00:06.165 INFO:teuthology.orchestra.run.smithi007.stdout:1710 trunc from 0x8324e47 to 0x7f315e9 2024-09-14T07:00:06.178 INFO:teuthology.orchestra.run.smithi007.stdout:1711 writesame 0x961f0bd thru 0x9620fd2 data_size 0x17(0x1f16 bytes) 2024-09-14T07:00:06.180 INFO:teuthology.orchestra.run.smithi007.stdout:1712 write 0x6da880e thru 0x6db533c (0xcb2f bytes) 2024-09-14T07:00:06.182 INFO:teuthology.orchestra.run.smithi007.stdout:1715 read 0x80f333f thru 0x80f8aee (0x57b0 bytes) 2024-09-14T07:00:06.182 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.182 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:06.181+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:1717 write 0x994bf1e thru 0x994c9a4 (0xa87 bytes) 2024-09-14T07:00:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:1718 read 0x81046e6 thru 0x810880c (0x4127 bytes) 2024-09-14T07:00:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:1720 punch from 0x1a9b45 to 0x1b68c5, (0xcd80 bytes) 2024-09-14T07:00:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:1722 punch from 0x9385707 to 0x9391abf, (0xc3b8 bytes) 2024-09-14T07:00:06.189 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.192 INFO:teuthology.orchestra.run.smithi007.stdout:1725 trunc from 0x994c9a5 to 0xc67a0d7 2024-09-14T07:00:06.197 INFO:teuthology.orchestra.run.smithi007.stdout:1726 read 0x7c14d85 thru 0x7c23cce (0xef4a bytes) 2024-09-14T07:00:06.198 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.198 INFO:teuthology.orchestra.run.smithi007.stdout:1728 write 0x32d77b thru 0x3395e3 (0xbe69 bytes) 2024-09-14T07:00:06.200 INFO:teuthology.orchestra.run.smithi007.stdout:1729 writesame 0x5e20546 thru 0x5e23370 data_size 0x1(0x2e2b bytes) 2024-09-14T07:00:06.211 INFO:teuthology.orchestra.run.smithi007.stdout:1730 writesame 0xd0d153a thru 0xd0db80c data_size 0x1(0xa2d3 bytes) 2024-09-14T07:00:06.214 INFO:teuthology.orchestra.run.smithi007.stdout:1731 trunc from 0xd0db80d to 0xedbf418 2024-09-14T07:00:06.219 INFO:teuthology.orchestra.run.smithi007.stdout:1735 punch from 0xd51a028 to 0xd51a9a5, (0x97d bytes) 2024-09-14T07:00:06.219 INFO:teuthology.orchestra.run.smithi007.stdout:1737 read 0x419b6f0 thru 0x41aa880 (0xf191 bytes) 2024-09-14T07:00:06.219 INFO:teuthology.orchestra.run.smithi007.stdout:1738 read 0x2a17f0f thru 0x2a27df4 (0xfee6 bytes) 2024-09-14T07:00:06.220 INFO:teuthology.orchestra.run.smithi007.stdout:1739 writesame 0x8531c69 thru 0x8533101 data_size 0x1(0x1499 bytes) 2024-09-14T07:00:06.223 INFO:teuthology.orchestra.run.smithi007.stdout:1742 read 0x3619f77 thru 0x362321a (0x92a4 bytes) 2024-09-14T07:00:06.224 INFO:teuthology.orchestra.run.smithi007.stdout:1743 write 0xafe71d2 thru 0xaff5537 (0xe366 bytes) 2024-09-14T07:00:06.226 INFO:teuthology.orchestra.run.smithi007.stdout:1745 read 0xed8d7c0 thru 0xed98c6d (0xb4ae bytes) 2024-09-14T07:00:06.226 INFO:teuthology.orchestra.run.smithi007.stdout:1746 write 0x65d036f thru 0x65d2352 (0x1fe4 bytes) 2024-09-14T07:00:06.228 INFO:teuthology.orchestra.run.smithi007.stdout:1747 punch from 0x66ebfb5 to 0x66f7aba, (0xbb05 bytes) 2024-09-14T07:00:06.228 INFO:teuthology.orchestra.run.smithi007.stdout:1749 write 0x6eccd77 thru 0x6ed7a6f (0xacf9 bytes) 2024-09-14T07:00:06.229 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.229 INFO:teuthology.orchestra.run.smithi007.stdout:1751 read 0x3d1000d thru 0x3d1be36 (0xbe2a bytes) 2024-09-14T07:00:06.230 INFO:teuthology.orchestra.run.smithi007.stdout:1752 writesame 0x7614538 thru 0x7617994 data_size 0x7(0x345d bytes) 2024-09-14T07:00:06.242 INFO:teuthology.orchestra.run.smithi007.stdout:1753 trunc from 0xedbf418 to 0x52db699 2024-09-14T07:00:06.249 INFO:teuthology.orchestra.run.smithi007.stdout:1754 write 0xf90f80 thru 0xf945ba (0x363b bytes) 2024-09-14T07:00:06.254 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:06.254 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:06.262 INFO:teuthology.orchestra.run.smithi007.stdout:1755 writesame 0xab28764 thru 0xab3178b data_size 0x1c(0x9028 bytes) 2024-09-14T07:00:06.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:06.273 INFO:teuthology.orchestra.run.smithi007.stdout:1756 write 0xed1a4c2 thru 0xed224ad (0x7fec bytes) 2024-09-14T07:00:06.276 INFO:teuthology.orchestra.run.smithi007.stdout:1757 write 0xf1c96b thru 0xf1cf16 (0x5ac bytes) 2024-09-14T07:00:06.278 INFO:teuthology.orchestra.run.smithi007.stdout:1759 read 0xa1e76ca thru 0xa1e94cd (0x1e04 bytes) 2024-09-14T07:00:06.278 INFO:teuthology.orchestra.run.smithi007.stdout:1760 write 0xd5d1ae7 thru 0xd5d4ffc (0x3516 bytes) 2024-09-14T07:00:06.279 INFO:teuthology.orchestra.run.smithi007.stdout:1761 write 0x345a4db thru 0x3460ed6 (0x69fc bytes) 2024-09-14T07:00:06.281 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.281 INFO:teuthology.orchestra.run.smithi007.stdout:1763 read 0xe12a2f2 thru 0xe12be57 (0x1b66 bytes) 2024-09-14T07:00:06.281 INFO:teuthology.orchestra.run.smithi007.stdout:1764 write 0x550462d thru 0x5510510 (0xbee4 bytes) 2024-09-14T07:00:06.283 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.283 INFO:teuthology.orchestra.run.smithi007.stdout:1766 write 0x5569978 thru 0x5570985 (0x700e bytes) 2024-09-14T07:00:06.283 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:06.282+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:06.284 INFO:teuthology.orchestra.run.smithi007.stdout:1767 read 0x8aa6ea4 thru 0x8aad1be (0x631b bytes) 2024-09-14T07:00:06.285 INFO:teuthology.orchestra.run.smithi007.stdout:1768 writesame 0x487c765 thru 0x4888c10 data_size 0x52(0xc4ac bytes) 2024-09-14T07:00:06.288 INFO:teuthology.orchestra.run.smithi007.stdout:1769 write 0xe440ed1 thru 0xe450b7b (0xfcab bytes) 2024-09-14T07:00:06.291 INFO:teuthology.orchestra.run.smithi007.stdout:1773 trunc from 0xed224ae to 0xca3a993 2024-09-14T07:00:06.302 INFO:teuthology.orchestra.run.smithi007.stdout:1774 write 0x350fc6f thru 0x3513a4d (0x3ddf bytes) 2024-09-14T07:00:06.304 INFO:teuthology.orchestra.run.smithi007.stdout:1775 write 0x19b30b9 thru 0x19ba47d (0x73c5 bytes) 2024-09-14T07:00:06.307 INFO:teuthology.orchestra.run.smithi007.stdout:1776 trunc from 0xca3a993 to 0xdaf7206 2024-09-14T07:00:06.312 INFO:teuthology.orchestra.run.smithi007.stdout:1777 write 0x59d2871 thru 0x59ddf14 (0xb6a4 bytes) 2024-09-14T07:00:06.314 INFO:teuthology.orchestra.run.smithi007.stdout:1782 write 0xd5f78bf thru 0xd5f8478 (0xbba bytes) 2024-09-14T07:00:06.316 INFO:teuthology.orchestra.run.smithi007.stdout:1783 read 0x38d53a1 thru 0x38d8fd0 (0x3c30 bytes) 2024-09-14T07:00:06.330 INFO:teuthology.orchestra.run.smithi007.stdout:1784 trunc from 0xdaf7206 to 0x113be2c 2024-09-14T07:00:06.349 INFO:teuthology.orchestra.run.smithi007.stdout:1786 write 0x66576a8 thru 0x665b6a1 (0x3ffa bytes) 2024-09-14T07:00:06.350 INFO:teuthology.orchestra.run.smithi007.stdout:1787 write 0x2d1a126 thru 0x2d1a572 (0x44d bytes) 2024-09-14T07:00:06.352 INFO:teuthology.orchestra.run.smithi007.stdout:1788 read 0x29a0888 thru 0x29adb2c (0xd2a5 bytes) 2024-09-14T07:00:06.352 INFO:teuthology.orchestra.run.smithi007.stdout:1790 punch from 0x2a3e6e to 0x2aca2f, (0x8bc1 bytes) 2024-09-14T07:00:06.352 INFO:teuthology.orchestra.run.smithi007.stdout:1792 read 0x5e47bed thru 0x5e4d8a6 (0x5cba bytes) 2024-09-14T07:00:06.361 INFO:teuthology.orchestra.run.smithi007.stdout:1793 writesame 0x90dd5f6 thru 0x90dd6b7 data_size 0x1(0xc2 bytes) 2024-09-14T07:00:06.362 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.373 INFO:teuthology.orchestra.run.smithi007.stdout:1795 write 0xdedb802 thru 0xdee5418 (0x9c17 bytes) 2024-09-14T07:00:06.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:06.374 INFO:teuthology.orchestra.run.smithi007.stdout:1798 punch from 0x6a8d5f1 to 0x6a9a094, (0xcaa3 bytes) 2024-09-14T07:00:06.375 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.375 INFO:teuthology.orchestra.run.smithi007.stdout:1800 write 0xeff1c8 thru 0xf0cbf6 (0xda2f bytes) 2024-09-14T07:00:06.376 INFO:teuthology.orchestra.run.smithi007.stdout:1802 punch from 0x7f9ba9f to 0x7fa81fd, (0xc75e bytes) 2024-09-14T07:00:06.377 INFO:teuthology.orchestra.run.smithi007.stdout:1804 read 0x7988373 thru 0x7998170 (0xfdfe bytes) 2024-09-14T07:00:06.384 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:06.383+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:06.384 INFO:teuthology.orchestra.run.smithi007.stdout:1806 trunc from 0xdee5419 to 0x76cf4f8 2024-09-14T07:00:06.392 INFO:teuthology.orchestra.run.smithi007.stdout:1807 writesame 0x249a9c3 thru 0x24a1f19 data_size 0x13(0x7557 bytes) 2024-09-14T07:00:06.394 INFO:teuthology.orchestra.run.smithi007.stdout:1809 read 0x56c44c2 thru 0x56d0c06 (0xc745 bytes) 2024-09-14T07:00:06.395 INFO:teuthology.orchestra.run.smithi007.stdout:1810 trunc from 0x76cf4f8 to 0x8374e57 2024-09-14T07:00:06.401 INFO:teuthology.orchestra.run.smithi007.stdout:1811 read 0x29108b3 thru 0x291551a (0x4c68 bytes) 2024-09-14T07:00:06.402 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:06.402 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T07:00:06.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:06.421 INFO:teuthology.orchestra.run.smithi007.stdout:1813 writesame 0xdaa4951 thru 0xdaa542b data_size 0x1(0xadb bytes) 2024-09-14T07:00:06.423 INFO:teuthology.orchestra.run.smithi007.stdout:1814 read 0x7fb4e36 thru 0x7fb7e07 (0x2fd2 bytes) 2024-09-14T07:00:06.423 INFO:teuthology.orchestra.run.smithi007.stdout:1815 punch from 0xbfcf8a1 to 0xbfdb19a, (0xb8f9 bytes) 2024-09-14T07:00:06.423 INFO:teuthology.orchestra.run.smithi007.stdout:1816 write 0x68128a8 thru 0x681b1e4 (0x893d bytes) 2024-09-14T07:00:06.425 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.427 INFO:teuthology.orchestra.run.smithi007.stdout:1819 trunc from 0xdaa542c to 0xbf0c6f8 2024-09-14T07:00:06.435 INFO:teuthology.orchestra.run.smithi007.stdout:1820 write 0x247d50d thru 0x24887f9 (0xb2ed bytes) 2024-09-14T07:00:06.439 INFO:teuthology.orchestra.run.smithi007.stdout:1823 read 0x319000f thru 0x319a867 (0xa859 bytes) 2024-09-14T07:00:06.440 INFO:teuthology.orchestra.run.smithi007.stdout:1824 read 0x3cfe4e9 thru 0x3d06ad3 (0x85eb bytes) 2024-09-14T07:00:06.440 INFO:teuthology.orchestra.run.smithi007.stdout:1825 read 0x4428eee thru 0x442f1e6 (0x62f9 bytes) 2024-09-14T07:00:06.441 INFO:teuthology.orchestra.run.smithi007.stdout:1827 writesame 0x4becc6d thru 0x4bf2b5d data_size 0x1(0x5ef1 bytes) 2024-09-14T07:00:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:1828 punch from 0x30a459c to 0x30b0412, (0xbe76 bytes) 2024-09-14T07:00:06.452 INFO:teuthology.orchestra.run.smithi007.stdout:1829 write 0xaa6e817 thru 0xaa74846 (0x6030 bytes) 2024-09-14T07:00:06.454 INFO:teuthology.orchestra.run.smithi007.stdout:1831 read 0x4ca456d thru 0x4cae206 (0x9c9a bytes) 2024-09-14T07:00:06.454 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.454 INFO:teuthology.orchestra.run.smithi007.stdout:1834 read 0x5ac0a59 thru 0x5ac52b1 (0x4859 bytes) 2024-09-14T07:00:06.455 INFO:teuthology.orchestra.run.smithi007.stdout:1836 punch from 0xb5ae870 to 0xb5afc47, (0x13d7 bytes) 2024-09-14T07:00:06.455 INFO:teuthology.orchestra.run.smithi007.stdout:1841 read 0x18feca6 thru 0x1900188 (0x14e3 bytes) 2024-09-14T07:00:06.455 INFO:teuthology.orchestra.run.smithi007.stdout:1843 write 0x5856122 thru 0x58637ec (0xd6cb bytes) 2024-09-14T07:00:06.457 INFO:teuthology.orchestra.run.smithi007.stdout:1844 punch from 0x6f9273f to 0x6f9f21b, (0xcadc bytes) 2024-09-14T07:00:06.457 INFO:teuthology.orchestra.run.smithi007.stdout:1846 read 0x1ab7419 thru 0x1ac0fe6 (0x9bce bytes) 2024-09-14T07:00:06.457 INFO:teuthology.orchestra.run.smithi007.stdout:1847 read 0x94a1341 thru 0x94a931d (0x7fdd bytes) 2024-09-14T07:00:06.464 INFO:teuthology.orchestra.run.smithi007.stdout:1848 write 0xc9e92f1 thru 0xc9f317e (0x9e8e bytes) 2024-09-14T07:00:06.466 INFO:teuthology.orchestra.run.smithi007.stdout:1849 punch from 0x9ff5a71 to 0x9ff97c8, (0x3d57 bytes) 2024-09-14T07:00:06.466 INFO:teuthology.orchestra.run.smithi007.stdout:1850 punch from 0x83fe56d to 0x8403596, (0x5029 bytes) 2024-09-14T07:00:06.466 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.466 INFO:teuthology.orchestra.run.smithi007.stdout:1852 write 0x6570c50 thru 0x6577997 (0x6d48 bytes) 2024-09-14T07:00:06.467 INFO:teuthology.orchestra.run.smithi007.stdout:1853 write 0xa0c22df thru 0xa0c663a (0x435c bytes) 2024-09-14T07:00:06.468 INFO:teuthology.orchestra.run.smithi007.stdout:1854 read 0x34c8a06 thru 0x34cda9d (0x5098 bytes) 2024-09-14T07:00:06.469 INFO:teuthology.orchestra.run.smithi007.stdout:1857 write 0xc9e27c4 thru 0xc9e33a2 (0xbdf bytes) 2024-09-14T07:00:06.470 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.470 INFO:teuthology.orchestra.run.smithi007.stdout:1859 read 0x5510d3d thru 0x551741a (0x66de bytes) 2024-09-14T07:00:06.470 INFO:teuthology.orchestra.run.smithi007.stdout:1861 read 0x5356ac4 thru 0x5361a24 (0xaf61 bytes) 2024-09-14T07:00:06.471 INFO:teuthology.orchestra.run.smithi007.stdout:1862 writesame 0x3eddb51 thru 0x3eebde9 data_size 0x1(0xe299 bytes) 2024-09-14T07:00:06.485 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:06.484+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:06.493 INFO:teuthology.orchestra.run.smithi007.stdout:1864 write 0x963f6f0 thru 0x964b1f9 (0xbb0a bytes) 2024-09-14T07:00:06.494 INFO:teuthology.orchestra.run.smithi007.stdout:1866 write 0xb3b90da thru 0xb3c291e (0x9845 bytes) 2024-09-14T07:00:06.496 INFO:teuthology.orchestra.run.smithi007.stdout:1868 write 0x53f5d1d thru 0x53ff85a (0x9b3e bytes) 2024-09-14T07:00:06.500 INFO:teuthology.orchestra.run.smithi007.stdout:1869 trunc from 0xc9f317f to 0xb0abd68 2024-09-14T07:00:06.510 INFO:teuthology.orchestra.run.smithi007.stdout:1871 read 0x7ae80d4 thru 0x7af800a (0xff37 bytes) 2024-09-14T07:00:06.510 INFO:teuthology.orchestra.run.smithi007.stdout:1872 read 0x818298b thru 0x81884fe (0x5b74 bytes) 2024-09-14T07:00:06.511 INFO:teuthology.orchestra.run.smithi007.stdout:1873 read 0x2cdd327 thru 0x2ce1974 (0x464e bytes) 2024-09-14T07:00:06.512 INFO:teuthology.orchestra.run.smithi007.stdout:1874 read 0x9c4e016 thru 0x9c59010 (0xaffb bytes) 2024-09-14T07:00:06.512 INFO:teuthology.orchestra.run.smithi007.stdout:1876 read 0x2d71a3e thru 0x2d78b56 (0x7119 bytes) 2024-09-14T07:00:06.512 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.513 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.519 INFO:teuthology.orchestra.run.smithi007.stdout:1879 trunc from 0xb0abd68 to 0x4b6dfd2 2024-09-14T07:00:06.522 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:06.532 INFO:teuthology.orchestra.run.smithi007.stdout:1880 trunc from 0x4b6dfd2 to 0x9c12714 2024-09-14T07:00:06.539 INFO:teuthology.orchestra.run.smithi007.stdout:1881 read 0x7e66720 thru 0x7e709e5 (0xa2c6 bytes) 2024-09-14T07:00:06.539 INFO:teuthology.orchestra.run.smithi007.stdout:1882 write 0x1467c1c thru 0x146f4c8 (0x78ad bytes) 2024-09-14T07:00:06.541 INFO:teuthology.orchestra.run.smithi007.stdout:1883 read 0x591073d thru 0x591625b (0x5b1f bytes) 2024-09-14T07:00:06.541 INFO:teuthology.orchestra.run.smithi007.stdout:1884 write 0x3cd4d82 thru 0x3cd6a4a (0x1cc9 bytes) 2024-09-14T07:00:06.543 INFO:teuthology.orchestra.run.smithi007.stdout:1885 read 0x55c15f0 thru 0x55c2468 (0xe79 bytes) 2024-09-14T07:00:06.545 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:06.545 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:06.548 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:06.548 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:06.552 INFO:teuthology.orchestra.run.smithi007.stdout:1886 write 0xcc06d34 thru 0xcc10072 (0x933f bytes) 2024-09-14T07:00:06.554 INFO:teuthology.orchestra.run.smithi007.stdout:1887 write 0x193fab thru 0x19f19d (0xb1f3 bytes) 2024-09-14T07:00:06.555 INFO:teuthology.orchestra.run.smithi007.stdout:1888 write 0xafd8500 thru 0xafdae4f (0x2950 bytes) 2024-09-14T07:00:06.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:06.558 INFO:teuthology.orchestra.run.smithi007.stdout:1890 read 0x15a1a2a thru 0x15a384e (0x1e25 bytes) 2024-09-14T07:00:06.558 INFO:teuthology.orchestra.run.smithi007.stdout:1891 punch from 0xb812d82 to 0xb81ea9e, (0xbd1c bytes) 2024-09-14T07:00:06.558 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.558 INFO:teuthology.orchestra.run.smithi007.stdout:1895 writesame 0xaa91a0b thru 0xaa9b812 data_size 0x34(0x9e08 bytes) 2024-09-14T07:00:06.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:06.565 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:06.567 INFO:teuthology.orchestra.run.smithi007.stdout:1898 trunc from 0xcc10073 to 0x63948a2 2024-09-14T07:00:06.575 INFO:teuthology.orchestra.run.smithi007.stdout:1900 write 0x13e7e1e thru 0x13f4f68 (0xd14b bytes) 2024-09-14T07:00:06.585 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:06.586 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:06.585+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:06.591 INFO:teuthology.orchestra.run.smithi007.stdout:1901 write 0xd405ded thru 0xd414823 (0xea37 bytes) 2024-09-14T07:00:06.593 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.600 INFO:teuthology.orchestra.run.smithi007.stdout:1904 write 0xe7e80ac thru 0xe7f1a4c (0x99a1 bytes) 2024-09-14T07:00:06.604 INFO:teuthology.orchestra.run.smithi007.stdout:1905 punch from 0x2ba5d29 to 0x2bac979, (0x6c50 bytes) 2024-09-14T07:00:06.604 INFO:teuthology.orchestra.run.smithi007.stdout:1906 write 0x1dfa1ca thru 0x1dfd03e (0x2e75 bytes) 2024-09-14T07:00:06.605 INFO:teuthology.orchestra.run.smithi007.stdout:1907 read 0xb4d8988 thru 0xb4de768 (0x5de1 bytes) 2024-09-14T07:00:06.606 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:06.606 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:06.606 INFO:teuthology.orchestra.run.smithi007.stdout:1908 writesame 0x7458bbe thru 0x7464797 data_size 0x3(0xbbda bytes) 2024-09-14T07:00:06.614 INFO:teuthology.orchestra.run.smithi007.stdout:1910 write 0x6fcd6d0 thru 0x6fdc5b0 (0xeee1 bytes) 2024-09-14T07:00:06.616 INFO:teuthology.orchestra.run.smithi007.stdout:1911 writesame 0x5d52178 thru 0x5d60511 data_size 0x53(0xe39a bytes) 2024-09-14T07:00:06.618 INFO:teuthology.orchestra.run.smithi007.stdout:1912 write 0x54c4489 thru 0x54d21dd (0xdd55 bytes) 2024-09-14T07:00:06.620 INFO:teuthology.orchestra.run.smithi007.stdout:1913 write 0x55fbc26 thru 0x5600f36 (0x5311 bytes) 2024-09-14T07:00:06.622 INFO:teuthology.orchestra.run.smithi007.stdout:1915 write 0x12e98a6 thru 0x12ee656 (0x4db1 bytes) 2024-09-14T07:00:06.623 INFO:teuthology.orchestra.run.smithi007.stdout:1916 writesame 0x52fb59e thru 0x52feacb data_size 0x3(0x352e bytes) 2024-09-14T07:00:06.626 INFO:teuthology.orchestra.run.smithi007.stdout:1917 read 0x6ca2f10 thru 0x6cab36c (0x845d bytes) 2024-09-14T07:00:06.626 INFO:teuthology.orchestra.run.smithi007.stdout:1918 write 0xbb90377 thru 0xbb99da3 (0x9a2d bytes) 2024-09-14T07:00:06.627 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:06.627 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:06.632 INFO:teuthology.orchestra.run.smithi007.stdout:1920 trunc from 0xe7f1a4d to 0xb188947 2024-09-14T07:00:06.644 INFO:teuthology.orchestra.run.smithi007.stdout:1921 write 0x7f5a184 thru 0x7f68960 (0xe7dd bytes) 2024-09-14T07:00:06.646 INFO:teuthology.orchestra.run.smithi007.stdout:1924 write 0x3ca300b thru 0x3cb02ef (0xd2e5 bytes) 2024-09-14T07:00:06.647 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:06.648 INFO:teuthology.orchestra.run.smithi007.stdout:1926 read 0x92ecfaa thru 0x92fab8a (0xdbe1 bytes) 2024-09-14T07:00:06.648 INFO:teuthology.orchestra.run.smithi007.stdout:1927 read 0x9f2faf7 thru 0x9f3a875 (0xad7f bytes) 2024-09-14T07:00:06.648 INFO:teuthology.orchestra.run.smithi007.stdout:1928 read 0x58d8084 thru 0x58dfc57 (0x7bd4 bytes) 2024-09-14T07:00:06.649 INFO:teuthology.orchestra.run.smithi007.stdout:1929 writesame 0x8d6e39 thru 0x8db079 data_size 0x1(0x4241 bytes) 2024-09-14T07:00:06.660 INFO:teuthology.orchestra.run.smithi007.stdout:1930 writesame 0x98bddc5 thru 0x98c425e data_size 0x2(0x649a bytes) 2024-09-14T07:00:06.665 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:06.665 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:06.667 INFO:teuthology.orchestra.run.smithi007.stdout:1931 writesame 0x57b3aae thru 0x57bf4df data_size 0x1(0xba32 bytes) 2024-09-14T07:00:06.687 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:06.686+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:06.689 INFO:teuthology.orchestra.run.smithi007.stdout:1932 read 0x6c1f17c thru 0x6c2194d (0x27d2 bytes) 2024-09-14T07:00:06.690 INFO:teuthology.orchestra.run.smithi007.stdout:1933 writesame 0x3ce8f6b thru 0x3cf3878 data_size 0x3(0xa90e bytes) 2024-09-14T07:00:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:06.693 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:06.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:06.707 INFO:teuthology.orchestra.run.smithi007.stdout:1934 write 0xe93a242 thru 0xe940a2d (0x67ec bytes) 2024-09-14T07:00:06.710 INFO:teuthology.orchestra.run.smithi007.stdout:1936 write 0xe8b87db thru 0xe8bc08c (0x38b2 bytes) 2024-09-14T07:00:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.713 INFO:teuthology.orchestra.run.smithi007.stdout:1939 read 0x7dfd6ec thru 0x7dff918 (0x222d bytes) 2024-09-14T07:00:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:1940 punch from 0xd707762 to 0xd70a4d4, (0x2d72 bytes) 2024-09-14T07:00:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:1941 read 0x5f0d40b thru 0x5f0ebeb (0x17e1 bytes) 2024-09-14T07:00:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:1942 read 0x3496cab thru 0x3497575 (0x8cb bytes) 2024-09-14T07:00:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:1945 read 0x87513bb thru 0x875a6a2 (0x92e8 bytes) 2024-09-14T07:00:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:1947 punch from 0xc157dc5 to 0xc16110d, (0x9348 bytes) 2024-09-14T07:00:06.715 INFO:teuthology.orchestra.run.smithi007.stdout:1948 read 0x1b88135 thru 0x1b9251b (0xa3e7 bytes) 2024-09-14T07:00:06.725 INFO:teuthology.orchestra.run.smithi007.stdout:1949 trunc from 0xe940a2e to 0x60ee01d 2024-09-14T07:00:06.736 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.739 INFO:teuthology.orchestra.run.smithi007.stdout:1951 trunc from 0x60ee01d to 0x40c8d04 2024-09-14T07:00:06.746 INFO:teuthology.orchestra.run.smithi007.stdout:1952 read 0x40a292f thru 0x40b1ad9 (0xf1ab bytes) 2024-09-14T07:00:06.758 INFO:teuthology.orchestra.run.smithi007.stdout:1953 trunc from 0x40c8d04 to 0xd85b088 2024-09-14T07:00:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:1954 punch from 0x3f4a11b to 0x3f51b67, (0x7a4c bytes) 2024-09-14T07:00:06.763 INFO:teuthology.orchestra.run.smithi007.stdout:1955 write 0x9e63258 thru 0x9e671cf (0x3f78 bytes) 2024-09-14T07:00:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:1956 punch from 0x7e15c73 to 0x7e15f2c, (0x2b9 bytes) 2024-09-14T07:00:06.765 INFO:teuthology.orchestra.run.smithi007.stdout:1957 write 0x93f78a1 thru 0x9402a12 (0xb172 bytes) 2024-09-14T07:00:06.767 INFO:teuthology.orchestra.run.smithi007.stdout:1958 writesame 0x7846d7d thru 0x78495fb data_size 0x1(0x287f bytes) 2024-09-14T07:00:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:1960 punch from 0x228106b to 0x22857e5, (0x477a bytes) 2024-09-14T07:00:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:1961 punch from 0xd390196 to 0xd39edc2, (0xec2c bytes) 2024-09-14T07:00:06.772 INFO:teuthology.orchestra.run.smithi007.stdout:1962 write 0x1cb235e thru 0x1cb7c0f (0x58b2 bytes) 2024-09-14T07:00:06.774 INFO:teuthology.orchestra.run.smithi007.stdout:1964 writesame 0x988e436 thru 0x98992f9 data_size 0xa(0xaec4 bytes) 2024-09-14T07:00:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:1965 writesame 0xa82262f thru 0xa82c5c1 data_size 0x59(0x9f93 bytes) 2024-09-14T07:00:06.778 INFO:teuthology.orchestra.run.smithi007.stdout:1966 read 0x5ed2d98 thru 0x5ede623 (0xb88c bytes) 2024-09-14T07:00:06.784 INFO:teuthology.orchestra.run.smithi007.stdout:1967 trunc from 0xd85b088 to 0x883e782 2024-09-14T07:00:06.788 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:06.787+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:06.790 INFO:teuthology.orchestra.run.smithi007.stdout:1968 read 0x23e2dfe thru 0x23ece91 (0xa094 bytes) 2024-09-14T07:00:06.791 INFO:teuthology.orchestra.run.smithi007.stdout:1970 read 0x84633f3 thru 0x846f8e6 (0xc4f4 bytes) 2024-09-14T07:00:06.798 INFO:teuthology.orchestra.run.smithi007.stdout:1971 write 0xa1ba3a1 thru 0xa1bcacb (0x272b bytes) 2024-09-14T07:00:06.800 INFO:teuthology.orchestra.run.smithi007.stdout:1972 writesame 0x9a56623 thru 0x9a58979 data_size 0x1(0x2357 bytes) 2024-09-14T07:00:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.804 INFO:teuthology.orchestra.run.smithi007.stdout:1975 write 0x34e9f55 thru 0x34f05a3 (0x664f bytes) 2024-09-14T07:00:06.805 INFO:teuthology.orchestra.run.smithi007.stdout:1978 writesame 0x582f999 thru 0x583bf01 data_size 0x1(0xc569 bytes) 2024-09-14T07:00:06.827 INFO:teuthology.orchestra.run.smithi007.stdout:1979 punch from 0x8a10ace to 0x8a1e569, (0xda9b bytes) 2024-09-14T07:00:06.827 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.827 INFO:teuthology.orchestra.run.smithi007.stdout:1981 write 0x2c6037 thru 0x2d0813 (0xa7dd bytes) 2024-09-14T07:00:06.828 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:06.829 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:06.829 INFO:teuthology.orchestra.run.smithi007.stdout:1982 read 0xcf6d89 thru 0xcfafcf (0x4247 bytes) 2024-09-14T07:00:06.835 INFO:teuthology.orchestra.run.smithi007.stdout:1983 writesame 0xb06c5e8 thru 0xb079f4c data_size 0xd(0xd965 bytes) 2024-09-14T07:00:06.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:06.842 INFO:teuthology.orchestra.run.smithi007.stdout:1984 write 0xb573184 thru 0xb57491a (0x1797 bytes) 2024-09-14T07:00:06.844 INFO:teuthology.orchestra.run.smithi007.stdout:1985 writesame 0x48eb121 thru 0x48f5bae data_size 0x39(0xaa8e bytes) 2024-09-14T07:00:06.845 INFO:teuthology.orchestra.run.smithi007.stdout:1987 write 0x15b1b1b thru 0x15ba146 (0x862c bytes) 2024-09-14T07:00:06.847 INFO:teuthology.orchestra.run.smithi007.stdout:1988 punch from 0x85d4c31 to 0x85e36f7, (0xeac6 bytes) 2024-09-14T07:00:06.847 INFO:teuthology.orchestra.run.smithi007.stdout:1989 punch from 0x2313096 to 0x23207cf, (0xd739 bytes) 2024-09-14T07:00:06.847 INFO:teuthology.orchestra.run.smithi007.stdout:1991 write 0x36f5863 thru 0x36fda9d (0x823b bytes) 2024-09-14T07:00:06.848 INFO:teuthology.orchestra.run.smithi007.stdout:1992 write 0x32fa3d3 thru 0x3305993 (0xb5c1 bytes) 2024-09-14T07:00:06.850 INFO:teuthology.orchestra.run.smithi007.stdout:1995 trunc from 0xb57491b to 0xb859e00 2024-09-14T07:00:06.856 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.865 INFO:teuthology.orchestra.run.smithi007.stdout:1998 write 0xe345caa thru 0xe348a74 (0x2dcb bytes) 2024-09-14T07:00:06.868 INFO:teuthology.orchestra.run.smithi007.stdout:1999 write 0xd1923a thru 0xd2059c (0x7363 bytes) 2024-09-14T07:00:06.870 INFO:teuthology.orchestra.run.smithi007.stdout:2000 writesame 0x71f7efd thru 0x7206097 data_size 0x1(0xe19b bytes) 2024-09-14T07:00:06.885 INFO:teuthology.orchestra.run.smithi007.stdout:2001 writesame 0x51ce0cf thru 0x51d43d0 data_size 0x2e(0x6302 bytes) 2024-09-14T07:00:06.887 INFO:teuthology.orchestra.run.smithi007.stdout:2002 read 0x8485cfd thru 0x8492666 (0xc96a bytes) 2024-09-14T07:00:06.888 INFO:teuthology.orchestra.run.smithi007.stdout:2003 write 0xcda51a7 thru 0xcdb0c05 (0xba5f bytes) 2024-09-14T07:00:06.888 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:06.887+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:06.889 INFO:teuthology.orchestra.run.smithi007.stdout:2004 read 0x60291c thru 0x608982 (0x6067 bytes) 2024-09-14T07:00:06.890 INFO:teuthology.orchestra.run.smithi007.stdout:2007 read 0x54174d7 thru 0x5424a27 (0xd551 bytes) 2024-09-14T07:00:06.890 INFO:teuthology.orchestra.run.smithi007.stdout:2008 read 0xa7f726c thru 0xa7fe84a (0x75df bytes) 2024-09-14T07:00:06.890 INFO:teuthology.orchestra.run.smithi007.stdout:2010 punch from 0x4cb6a93 to 0x4cc59bc, (0xef29 bytes) 2024-09-14T07:00:06.891 INFO:teuthology.orchestra.run.smithi007.stdout:2012 read 0x3aedb1d thru 0x3afb4c4 (0xd9a8 bytes) 2024-09-14T07:00:06.891 INFO:teuthology.orchestra.run.smithi007.stdout:2013 read 0xcd1254f thru 0xcd1ba3a (0x94ec bytes) 2024-09-14T07:00:06.896 INFO:teuthology.orchestra.run.smithi007.stdout:2014 trunc from 0xe348a75 to 0xa358d60 2024-09-14T07:00:06.906 INFO:teuthology.orchestra.run.smithi007.stdout:2015 write 0x27dbb24 thru 0x27df0c7 (0x35a4 bytes) 2024-09-14T07:00:06.911 INFO:teuthology.orchestra.run.smithi007.stdout:2016 trunc from 0xa358d60 to 0x7db3308 2024-09-14T07:00:06.918 INFO:teuthology.orchestra.run.smithi007.stdout:2017 read 0x14d1462 thru 0x14d4fad (0x3b4c bytes) 2024-09-14T07:00:06.925 INFO:teuthology.orchestra.run.smithi007.stdout:2018 trunc from 0x7db3308 to 0x1672fb0 2024-09-14T07:00:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:2020 read 0xccfce0 thru 0xcd3542 (0x3863 bytes) 2024-09-14T07:00:06.940 INFO:teuthology.orchestra.run.smithi007.stdout:2021 write 0x1f3e9c4 thru 0x1f40b24 (0x2161 bytes) 2024-09-14T07:00:06.950 INFO:teuthology.orchestra.run.smithi007.stdout:2023 write 0x3d6350c thru 0x3d6def4 (0xa9e9 bytes) 2024-09-14T07:00:06.954 INFO:teuthology.orchestra.run.smithi007.stdout:2024 trunc from 0x3d6def5 to 0x5ffe5c3 2024-09-14T07:00:06.959 INFO:teuthology.orchestra.run.smithi007.stdout:2025 read 0x1427e09 thru 0x14379ca (0xfbc2 bytes) 2024-09-14T07:00:06.960 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.213432269000002, 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:00:06.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:00:06.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:06.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.615019676999999, 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:06.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:00:06.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:00:06.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:06.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.982 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.609360034000002, 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:00:06.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:00:06.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:00:06.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:00:06.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:00:06.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.303523756000001, 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:06.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:00:06.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:06.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:06.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.303492964, 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:00:06.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.303469593999999, 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:06.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:06.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:00:07.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:00:07.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.302663084999999, 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:00:07.002 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.302640692000001, 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:07.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.302620552, 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:07.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:00:07.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.302470397, 2024-09-14T07:00:07.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.302414843000001, 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.302381801999999, 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.300028717, 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:00:07.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.300006152000002, 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:00:07.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45774 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 933888~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.299169415000001, 2024-09-14T07:00:07.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068647313000000001, 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45774 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668062+0000", 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668064+0000", 2024-09-14T07:00:07.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.058e-06 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668070+0000", 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668075+0000", 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7859999999999996e-06 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736553+0000", 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068478400999999994 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736622+0000", 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8732999999999994e-05 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736709+0000", 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6453999999999996e-05 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.041 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.299149156999999, 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:00:07.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:07.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.297932136, 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.046 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:07.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:00:07.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:00:07.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.297804295999999, 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:00:07.050 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:00:07.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.297784202999999, 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.055 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:07.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.29675731, 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:07.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:07.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.063 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:07.065 INFO:teuthology.orchestra.run.smithi007.stdout:2029 write 0xbc15ab0 thru 0xbc1f291 (0x97e2 bytes) 2024-09-14T07:00:07.065 INFO:teuthology.orchestra.run.smithi007.stdout:2030 write 0xbb4e0a7 thru 0xbb4f0d7 (0x1031 bytes) 2024-09-14T07:00:07.065 INFO:teuthology.orchestra.run.smithi007.stdout:2031 punch from 0x779f8cf to 0x77a59a8, (0x60d9 bytes) 2024-09-14T07:00:07.065 INFO:teuthology.orchestra.run.smithi007.stdout:2032 punch from 0x173f8eb to 0x174c0f8, (0xc80d bytes) 2024-09-14T07:00:07.065 INFO:teuthology.orchestra.run.smithi007.stdout:2033 write 0xb4f9abb thru 0xb50443b (0xa981 bytes) 2024-09-14T07:00:07.065 INFO:teuthology.orchestra.run.smithi007.stdout:2035 writesame 0x27ffc7f thru 0x2800125 data_size 0x1(0x4a7 bytes) 2024-09-14T07:00:07.066 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:07.069 INFO:teuthology.orchestra.run.smithi007.stdout:2036 write 0x18b8752 thru 0x18c3c8e (0xb53d bytes) 2024-09-14T07:00:07.069 INFO:teuthology.orchestra.run.smithi007.stdout:2038 write 0x772d5fe thru 0x7739cac (0xc6af bytes) 2024-09-14T07:00:07.069 INFO:teuthology.orchestra.run.smithi007.stdout:2041 writesame 0xae7cfbe thru 0xae8282d data_size 0x5(0x5870 bytes) 2024-09-14T07:00:07.069 INFO:teuthology.orchestra.run.smithi007.stdout:2043 punch from 0x5637e6a to 0x5639cc7, (0x1e5d bytes) 2024-09-14T07:00:07.069 INFO:teuthology.orchestra.run.smithi007.stdout:2046 write 0xb2fc5f3 thru 0xb301c5e (0x566c bytes) 2024-09-14T07:00:07.069 INFO:teuthology.orchestra.run.smithi007.stdout:2047 write 0x2213829 thru 0x2222700 (0xeed8 bytes) 2024-09-14T07:00:07.069 INFO:teuthology.orchestra.run.smithi007.stdout:2048 read 0x4c8cd81 thru 0x4c9c0fb (0xf37b bytes) 2024-09-14T07:00:07.069 INFO:teuthology.orchestra.run.smithi007.stdout:2050 punch from 0x4ffaceb to 0x4ffcb44, (0x1e59 bytes) 2024-09-14T07:00:07.069 INFO:teuthology.orchestra.run.smithi007.stdout:2052 trunc from 0xbc1f292 to 0x96e211 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2053 write 0x26148b8 thru 0x261f991 (0xb0da bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2056 punch from 0x123b458 to 0x1246478, (0xb020 bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2057 trunc from 0x261f992 to 0xde8d7b9 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2058 read 0x465a911 thru 0x46647bd (0x9ead bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2059 write 0x8d3da01 thru 0x8d4570f (0x7d0f bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2060 read 0x2ede3a1 thru 0x2ede53a (0x19a bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2061 write 0xeae73f thru 0xeb1ce4 (0x35a6 bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2062 punch from 0x969f060 to 0x96a7b25, (0x8ac5 bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2063 write 0x2e7b98a thru 0x2e7e893 (0x2f0a bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2064 write 0xe4a8e00 thru 0xe4ab81a (0x2a1b bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2065 read 0x32843db thru 0x328b843 (0x7469 bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2066 read 0xcbbb178 thru 0xcbca76d (0xf5f6 bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2069 read 0x7c488e7 thru 0x7c5485f (0xbf79 bytes) 2024-09-14T07:00:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:2071 write 0x9bfcafa thru 0x9c0bbd4 (0xf0db bytes) 2024-09-14T07:00:07.071 INFO:teuthology.orchestra.run.smithi007.stdout:2072 writesame 0x8346c59 thru 0x83475c3 data_size 0x1(0x96b bytes) 2024-09-14T07:00:07.071 INFO:teuthology.orchestra.run.smithi007.stdout:2073 punch from 0x45a23c1 to 0x45a5c58, (0x3897 bytes) 2024-09-14T07:00:07.071 INFO:teuthology.orchestra.run.smithi007.stdout:2074 write 0x5392bd thru 0x541b51 (0x8895 bytes) 2024-09-14T07:00:07.071 INFO:teuthology.orchestra.run.smithi007.stdout:2075 trunc from 0xe4ab81b to 0xb374549 2024-09-14T07:00:07.071 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:07.064+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:07.082 INFO:teuthology.orchestra.run.smithi007.stdout:2076 trunc from 0xb374549 to 0x2aa84a3 2024-09-14T07:00:07.091 INFO:teuthology.orchestra.run.smithi007.stdout:2077 punch from 0x9aab09 to 0x9ac0ff, (0x15f6 bytes) 2024-09-14T07:00:07.091 INFO:teuthology.orchestra.run.smithi007.stdout:2078 read 0x25c9d31 thru 0x25d4ddf (0xb0af bytes) 2024-09-14T07:00:07.106 INFO:teuthology.orchestra.run.smithi007.stdout:2079 write 0xb156ad3 thru 0xb163e64 (0xd392 bytes) 2024-09-14T07:00:07.108 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:07.108 INFO:teuthology.orchestra.run.smithi007.stdout:2083 write 0x4805013 thru 0x480f393 (0xa381 bytes) 2024-09-14T07:00:07.123 INFO:teuthology.orchestra.run.smithi007.stdout:2085 trunc from 0xb163e65 to 0x10c20f 2024-09-14T07:00:07.135 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:07.152 INFO:teuthology.orchestra.run.smithi007.stdout:2088 write 0xa7013e8 thru 0xa70e424 (0xd03d bytes) 2024-09-14T07:00:07.154 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:07.154 INFO:teuthology.orchestra.run.smithi007.stdout:2091 read 0x1bc26bb thru 0x1bce3bd (0xbd03 bytes) 2024-09-14T07:00:07.161 INFO:teuthology.orchestra.run.smithi007.stdout:2092 trunc from 0xa70e425 to 0x4565594 2024-09-14T07:00:07.165 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:07.165+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:07.179 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:07.179 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:07.179 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:07.179 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:07.181 INFO:teuthology.orchestra.run.smithi007.stdout:2093 write 0xbf4b9c7 thru 0xbf59171 (0xd7ab bytes) 2024-09-14T07:00:07.183 INFO:teuthology.orchestra.run.smithi007.stdout:2095 trunc from 0xbf59172 to 0xbe06ccf 2024-09-14T07:00:07.188 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:07.194 INFO:teuthology.orchestra.run.smithi007.stdout:2096 trunc from 0xbe06ccf to 0x7bed094 2024-09-14T07:00:07.201 INFO:teuthology.orchestra.run.smithi007.stdout:2099 write 0x31c26b0 thru 0x31cde04 (0xb755 bytes) 2024-09-14T07:00:07.203 INFO:teuthology.orchestra.run.smithi007.stdout:2100 write 0x4d622f0 thru 0x4d697ec (0x74fd bytes) 2024-09-14T07:00:07.205 INFO:teuthology.orchestra.run.smithi007.stdout:2101 writesame 0x56cf820 thru 0x56d9beb data_size 0x16(0xa3cc bytes) 2024-09-14T07:00:07.214 INFO:teuthology.orchestra.run.smithi007.stdout:2102 writesame 0x9c5d393 thru 0x9c6b504 data_size 0x3(0xe172 bytes) 2024-09-14T07:00:07.215 INFO:teuthology.orchestra.run.smithi007.stdout:2103 read 0x7d82fb5 thru 0x7d90ec4 (0xdf10 bytes) 2024-09-14T07:00:07.215 INFO:teuthology.orchestra.run.smithi007.stdout:2107 write 0x57ec08a thru 0x57fa8ee (0xe865 bytes) 2024-09-14T07:00:07.217 INFO:teuthology.orchestra.run.smithi007.stdout:2108 punch from 0x3d988d9 to 0x3da2b44, (0xa26b bytes) 2024-09-14T07:00:07.222 INFO:teuthology.orchestra.run.smithi007.stdout:2109 trunc from 0x9c6b505 to 0xe2c3cc6 2024-09-14T07:00:07.235 INFO:teuthology.orchestra.run.smithi007.stdout:2110 trunc from 0xe2c3cc6 to 0x760ac51 2024-09-14T07:00:07.241 INFO:teuthology.orchestra.run.smithi007.stdout:2112 clone 17 order 23 su 524288 sc 5 2024-09-14T07:00:07.267 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:07.266+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:07.292 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:07.292 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:07.292 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:07.292 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:07.292 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:07.300 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:07.368 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:07.367+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7619 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25436160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533626+0000", 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870996603999998, 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050835010999999999, 2024-09-14T07:00:07.406 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7619 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533626+0000", 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533626+0000", 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533627+0000", 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9499999999999998e-07 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533629+0000", 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.812e-06 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533634+0000", 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9529999999999996e-06 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533639+0000", 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6999999999999999e-06 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.583252+0000", 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049613288999999998 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.583328+0000", 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5112999999999997e-05 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584461+0000", 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0011338489999999999 2024-09-14T07:00:07.410 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7620 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25550848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533671+0000", 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870951605999998, 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050947723, 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.411 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7620 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533671+0000", 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533671+0000", 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533672+0000", 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9800000000000003e-07 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533675+0000", 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0139999999999999e-06 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533686+0000", 2024-09-14T07:00:07.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1104000000000001e-05 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533691+0000", 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9189999999999998e-06 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584498+0000", 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050806905999999999 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584547+0000", 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8733999999999998e-05 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584619+0000", 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2447999999999995e-05 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.415 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7621 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25665536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533699+0000", 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870923525999999, 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051134163000000003, 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7621 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533699+0000", 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533699+0000", 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533700+0000", 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0400000000000002e-07 2024-09-14T07:00:07.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533701+0000", 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4950000000000001e-06 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533705+0000", 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.788e-06 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533709+0000", 2024-09-14T07:00:07.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8419999999999998e-06 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584659+0000", 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050950016000000001 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584747+0000", 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8117999999999999e-05 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584834+0000", 2024-09-14T07:00:07.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6500000000000002e-05 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7622 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25780224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533713+0000", 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870909866000002, 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051311927, 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.420 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7622 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533713+0000", 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533713+0000", 2024-09-14T07:00:07.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533713+0000", 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.29e-07 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533716+0000", 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.362e-06 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533719+0000", 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3129999999999999e-06 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533722+0000", 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6259999999999999e-06 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584874+0000", 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051152345000000002 2024-09-14T07:00:07.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.584960+0000", 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5795000000000001e-05 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585025+0000", 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5257000000000002e-05 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7623 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 25894912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533725+0000", 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870897898999999, 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051497312000000003, 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7623 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533725+0000", 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533725+0000", 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533725+0000", 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3099999999999999e-07 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533726+0000", 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0109999999999999e-06 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533729+0000", 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.01e-06 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533732+0000", 2024-09-14T07:00:07.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.57e-06 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585061+0000", 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051328632999999999 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585144+0000", 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3061999999999999e-05 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585222+0000", 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8794999999999994e-05 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7624 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26009600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533735+0000", 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870888116, 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051626949999999998, 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.429 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7624 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533735+0000", 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533735+0000", 2024-09-14T07:00:07.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533913+0000", 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017811300000000001 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533918+0000", 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6820000000000002e-06 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533932+0000", 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3899e-05 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533939+0000", 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5190000000000003e-06 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585252+0000", 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051313304999999997 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585297+0000", 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4149e-05 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585362+0000", 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5283000000000005e-05 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.433 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7625 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26124288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.533952+0000", 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870671165000001, 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051605167, 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7625 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533952+0000", 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.533952+0000", 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534144+0000", 2024-09-14T07:00:07.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019176499999999999 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534147+0000", 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.382e-06 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534156+0000", 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2289999999999992e-06 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534161+0000", 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7960000000000003e-06 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585402+0000", 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051240648 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585484+0000", 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1897000000000005e-05 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585557+0000", 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3449999999999996e-05 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7626 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26238976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.534171+0000", 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870451719999998, 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051588410000000001, 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.438 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7626 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534171+0000", 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534171+0000", 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534297+0000", 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012619899999999999 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534305+0000", 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3329999999999999e-06 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534313+0000", 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4290000000000001e-06 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534320+0000", 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6780000000000003e-06 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585598+0000", 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051277572 2024-09-14T07:00:07.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585682+0000", 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4858000000000006e-05 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585760+0000", 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7341000000000004e-05 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7627 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26353664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.534330+0000", 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870292622000001, 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051630638, 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7627 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534330+0000", 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534330+0000", 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534576+0000", 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00024544699999999998 2024-09-14T07:00:07.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534581+0000", 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7870000000000001e-06 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534589+0000", 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6119999999999999e-06 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534596+0000", 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6590000000000001e-06 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585801+0000", 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051204972000000001 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585883+0000", 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2494e-05 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585961+0000", 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7667e-05 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7628 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 26468352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.534609+0000", 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.870014497, 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051474580999999998, 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.447 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7628 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534609+0000", 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.534609+0000", 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538869+0000", 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042609450000000004 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538874+0000", 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5050000000000001e-06 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538884+0000", 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0242e-05 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.538891+0000", 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2119999999999997e-06 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.585991+0000", 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.047099540000000002 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.586038+0000", 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6971000000000001e-05 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.586083+0000", 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5166000000000001e-05 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.451 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7644 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28319744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539166+0000", 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.865456593000001, 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049804237000000001, 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.452 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7644 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539166+0000", 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539166+0000", 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539167+0000", 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539168+0000", 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7100000000000011e-07 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539172+0000", 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0980000000000004e-06 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539175+0000", 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0319999999999999e-06 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.588864+0000", 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049688831000000003 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.588928+0000", 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4193e-05 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.588971+0000", 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2910000000000001e-05 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7645 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28434432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539179+0000", 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.865443590000002, 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04998959, 2024-09-14T07:00:07.456 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7645 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539179+0000", 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539179+0000", 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539180+0000", 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539182+0000", 2024-09-14T07:00:07.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4229999999999999e-06 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539189+0000", 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6270000000000002e-06 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539194+0000", 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4970000000000001e-06 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589004+0000", 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049809785000000002 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589091+0000", 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7348999999999993e-05 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589169+0000", 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7655000000000001e-05 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.460 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7646 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28549120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539202+0000", 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.865421303000002, 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050152584, 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.461 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7646 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539202+0000", 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539202+0000", 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539202+0000", 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6199999999999999e-07 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539203+0000", 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.218e-06 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539207+0000", 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2540000000000003e-06 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539210+0000", 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.757e-06 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589199+0000", 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.049988800999999999 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589275+0000", 2024-09-14T07:00:07.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6174000000000006e-05 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589354+0000", 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9117999999999997e-05 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7647 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28663808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539216+0000", 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.865406586999999, 2024-09-14T07:00:07.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050330617000000001, 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7647 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539216+0000", 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539216+0000", 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539217+0000", 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539218+0000", 2024-09-14T07:00:07.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6589999999999999e-06 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539225+0000", 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2859999999999996e-06 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539229+0000", 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3069999999999996e-06 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589391+0000", 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0501623 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589472+0000", 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0752000000000002e-05 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589547+0000", 2024-09-14T07:00:07.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5018999999999994e-05 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7648 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28778496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539236+0000", 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.865387318, 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050463816000000002, 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.470 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7648 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539236+0000", 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539236+0000", 2024-09-14T07:00:07.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539236+0000", 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539237+0000", 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.263e-06 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539241+0000", 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3960000000000002e-06 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539243+0000", 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7269999999999999e-06 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589586+0000", 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050343176000000003 2024-09-14T07:00:07.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589656+0000", 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9530999999999995e-05 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589700+0000", 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3511999999999999e-05 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7649 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 28893184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539247+0000", 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.865376446999999, 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050647407999999998, 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7649 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539247+0000", 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539247+0000", 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539247+0000", 2024-09-14T07:00:07.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539248+0000", 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0279999999999999e-06 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539251+0000", 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9019999999999999e-06 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539253+0000", 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2989999999999999e-06 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589733+0000", 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050480216000000001 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589817+0000", 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4159000000000005e-05 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589894+0000", 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6595999999999996e-05 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7650 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 29007872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539256+0000", 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.865367242000001, 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050806128999999998, 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.479 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7650 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539256+0000", 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539256+0000", 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539256+0000", 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9999999999999999e-07 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539261+0000", 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7820000000000001e-06 2024-09-14T07:00:07.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539265+0000", 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1219999999999997e-06 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539267+0000", 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5689999999999999e-06 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589925+0000", 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050657695000000003 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.589979+0000", 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4088999999999997e-05 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590062+0000", 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2671999999999997e-05 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7651 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 29122560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539271+0000", 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.865352199, 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050987034000000001, 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.484 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7651 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539271+0000", 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539271+0000", 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539271+0000", 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539272+0000", 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4099999999999997e-07 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539275+0000", 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0699999999999998e-06 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539278+0000", 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5179999999999999e-06 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590100+0000", 2024-09-14T07:00:07.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050822289 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590181+0000", 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1269999999999999e-05 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590258+0000", 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6737999999999996e-05 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7652 3.2 3:47e69009:::rbd_data.114f7dc3575d.0000000000000003:8 [read 29237248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.539280+0000", 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.865342672000001, 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.051152192999999999, 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7652 2024-09-14T07:00:07.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539280+0000", 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.539280+0000", 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543135+0000", 2024-09-14T07:00:07.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0038546919999999998 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543140+0000", 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543151+0000", 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1314e-05 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543156+0000", 2024-09-14T07:00:07.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9590000000000003e-06 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590297+0000", 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.047140973000000003 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590381+0000", 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4006999999999997e-05 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590433+0000", 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1456999999999999e-05 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:7654 3.2 3:41572fcc:::rbd_object_map.114f54fbf0ed:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.543182+0000", 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.861441062000001, 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.050091802999999997, 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:07.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 7654 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543182+0000", 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543182+0000", 2024-09-14T07:00:07.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543182+0000", 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6600000000000002e-07 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543184+0000", 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.063e-06 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543188+0000", 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8680000000000001e-06 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.543191+0000", 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.881e-06 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590653+0000", 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.047461934999999997 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.590706+0000", 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3340000000000001e-05 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.591041+0000", 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00033495900000000002 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593167+0000", 2024-09-14T07:00:07.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021259389999999999 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593183+0000", 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5515e-05 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593233+0000", 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9777000000000001e-05 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.593274+0000", 2024-09-14T07:00:07.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1159999999999999e-05 2024-09-14T07:00:07.499 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.499 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.499 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.499 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.499 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.499 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:07.500 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:07.501 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:07.499+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:07.531 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone16 from 0x760ac51 (overlap 0x10c20f) to 0xf150d 2024-09-14T07:00:07.541 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:07.541 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:07.541 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T07:00:07.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:00:07.601 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:07.600+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:07.607 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:07.607 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:07.607 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:07.607 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:07.616 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:07.699 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-14T07:00:07.701 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:07.701+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:07.769 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:07.769 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:07.769 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:07.769 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:07.770 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:07.780 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:07.824 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:07.823+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 56, 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6", 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-14T06:58:59.479055+0000", 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-14T07:00:07.407924+0000", 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-14T06:59:11.718934+0000", 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-14T06:59:50.317515+0000", 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5806080, 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-14T07:00:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 6, 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:00:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:11.765419+0000", 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:00:07.860 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:00:07.861 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "18", 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-14T07:00:07.862 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:00:07.863 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:00:07.864 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 6.9000000953674316, 2024-09-14T07:00:07.865 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 6.9000000953674316, 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:18.343475+0000", 2024-09-14T07:00:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-14T07:00:07.867 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "22", 2024-09-14T07:00:07.868 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 22, 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:00:07.869 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:00:07.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:00:07.871 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 4.380000114440918, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 4.380000114440918, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 4.380000114440918, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 4.380000114440918, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T07:00:07.872 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:27.257805+0000", 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:00:07.873 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:00:07.874 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:00:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:00:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:00:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:00:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:00:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "56", 2024-09-14T07:00:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:00:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:00:07.875 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 19, 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 56, 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:00:07.876 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:00:07.877 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:00:07.878 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.75, 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.75, 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.75, 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.75, 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.879 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c036038d-cffb-4584-8851-69604a31ee40", 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 51, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/2081065366", 2024-09-14T07:00:07.884 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/2081065366", 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/2081065366", 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/2081065366", 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "1c80080f-932a-4957-b64b-f30c4bdb9340", 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:07.885 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 50, 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6806/1688050909", 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6810/1688050909", 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/1688050909", 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/1688050909", 2024-09-14T07:00:07.890 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f3544d11-8a0f-498c-a03c-5d34134c9773", 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:07.891 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/1872259061", 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/1872259061", 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/1872259061", 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/1872259061", 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d2261455-c91b-414b-925f-388f0e094dbe", 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 44, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:07.897 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-14T07:00:07.898 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-14T07:00:07.899 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-14T07:00:07.900 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-14T07:00:07.901 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/1193301030", 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/1193301030", 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6811/1193301030", 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6807/1193301030", 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.902 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4088d23b-0d06-4415-9eba-6fb64355746c", 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6810", 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6811", 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6812", 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6813", 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6816", 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6817", 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6814", 2024-09-14T07:00:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6815", 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6811/2283562461", 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6813/2283562461", 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6817/2283562461", 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6815/2283562461", 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "5e27cb53-ecce-4061-bc5a-be599bf18cc4", 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:07.909 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6818", 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6819", 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.910 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6820", 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6821", 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.911 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6824", 2024-09-14T07:00:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6825", 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6822", 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6823", 2024-09-14T07:00:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6819/2212769626", 2024-09-14T07:00:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6821/2212769626", 2024-09-14T07:00:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6825/2212769626", 2024-09-14T07:00:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6823/2212769626", 2024-09-14T07:00:07.914 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T07:00:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ceb9a1f1-f54c-48d5-91d8-2287cac96ceb", 2024-09-14T07:00:07.915 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.916 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6826", 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6827", 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6828", 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6829", 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6832", 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6833", 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6830", 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6831", 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6827/3197231389", 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6829/3197231389", 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6833/3197231389", 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6831/3197231389", 2024-09-14T07:00:07.920 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85c59cd6-6e2b-4cc0-852d-d1f075a8826d", 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:07.921 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 41, 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6800", 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6801", 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6802", 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6803", 2024-09-14T07:00:07.923 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6806", 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6807", 2024-09-14T07:00:07.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6804", 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6805", 2024-09-14T07:00:07.925 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6801/2162534312", 2024-09-14T07:00:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6803/2162534312", 2024-09-14T07:00:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6807/2162534312", 2024-09-14T07:00:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6805/2162534312", 2024-09-14T07:00:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:07.926 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.064941+0000", 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.780598+0000", 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.127700+0000", 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:07.930 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.730995+0000", 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.864423+0000", 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.329361+0000", 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:07.932 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:10.107581+0000", 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:07.933 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.300043+0000", 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.934 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1f", 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 5, 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 0 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-14T07:00:07.935 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-14T07:00:07.936 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-14T07:00:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-14T07:00:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:00:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:00:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-14T07:00:07.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:07.937 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:07.940 INFO:tasks.thrashosds.thrasher:j is {'epoch': 56, 'fsid': '32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6', 'created': '2024-09-14T06:58:59.479055+0000', 'modified': '2024-09-14T07:00:07.407924+0000', 'last_up_change': '2024-09-14T06:59:11.718934+0000', 'last_in_change': '2024-09-14T06:59:50.317515+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:59:11.765419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:59:18.343475+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.380000114440918, 'score_stable': 4.380000114440918, 'optimal_score': 1, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:59:27.257805+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 56, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c036038d-cffb-4584-8851-69604a31ee40', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 2081065366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 2081065366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 2081065366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 2081065366}]}, 'public_addr': '172.21.15.7:6817/2081065366', 'cluster_addr': '172.21.15.7:6819/2081065366', 'heartbeat_back_addr': '172.21.15.7:6823/2081065366', 'heartbeat_front_addr': '172.21.15.7:6821/2081065366', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1c80080f-932a-4957-b64b-f30c4bdb9340', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 50, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6806', 'nonce': 1688050909}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6810', 'nonce': 1688050909}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 1688050909}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 1688050909}]}, 'public_addr': '172.21.15.7:6806/1688050909', 'cluster_addr': '172.21.15.7:6810/1688050909', 'heartbeat_back_addr': '172.21.15.7:6815/1688050909', 'heartbeat_front_addr': '172.21.15.7:6813/1688050909', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f3544d11-8a0f-498c-a03c-5d34134c9773', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 1872259061}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 1872259061}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 1872259061}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 1872259061}]}, 'public_addr': '172.21.15.7:6825/1872259061', 'cluster_addr': '172.21.15.7:6827/1872259061', 'heartbeat_back_addr': '172.21.15.7:6831/1872259061', 'heartbeat_front_addr': '172.21.15.7:6829/1872259061', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd2261455-c91b-414b-925f-388f0e094dbe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 1193301030}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 1193301030}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6809', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 1193301030}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6805', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 1193301030}]}, 'public_addr': '172.21.15.7:6801/1193301030', 'cluster_addr': '172.21.15.7:6803/1193301030', 'heartbeat_back_addr': '172.21.15.7:6811/1193301030', 'heartbeat_front_addr': '172.21.15.7:6807/1193301030', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4088d23b-0d06-4415-9eba-6fb64355746c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6810', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6811', 'nonce': 2283562461}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6812', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6813', 'nonce': 2283562461}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6816', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6817', 'nonce': 2283562461}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6814', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6815', 'nonce': 2283562461}]}, 'public_addr': '172.21.15.189:6811/2283562461', 'cluster_addr': '172.21.15.189:6813/2283562461', 'heartbeat_back_addr': '172.21.15.189:6817/2283562461', 'heartbeat_front_addr': '172.21.15.189:6815/2283562461', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5e27cb53-ecce-4061-bc5a-be599bf18cc4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6818', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6819', 'nonce': 2212769626}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6820', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6821', 'nonce': 2212769626}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6824', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6825', 'nonce': 2212769626}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6822', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6823', 'nonce': 2212769626}]}, 'public_addr': '172.21.15.189:6819/2212769626', 'cluster_addr': '172.21.15.189:6821/2212769626', 'heartbeat_back_addr': '172.21.15.189:6825/2212769626', 'heartbeat_front_addr': '172.21.15.189:6823/2212769626', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceb9a1f1-f54c-48d5-91d8-2287cac96ceb', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6826', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6827', 'nonce': 3197231389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6828', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6829', 'nonce': 3197231389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 3197231389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6830', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6831', 'nonce': 3197231389}]}, 'public_addr': '172.21.15.189:6827/3197231389', 'cluster_addr': '172.21.15.189:6829/3197231389', 'heartbeat_back_addr': '172.21.15.189:6833/3197231389', 'heartbeat_front_addr': '172.21.15.189:6831/3197231389', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '85c59cd6-6e2b-4cc0-852d-d1f075a8826d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6800', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6801', 'nonce': 2162534312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6802', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6803', 'nonce': 2162534312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6806', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6807', 'nonce': 2162534312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6804', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6805', 'nonce': 2162534312}]}, 'public_addr': '172.21.15.189:6801/2162534312', 'cluster_addr': '172.21.15.189:6803/2162534312', 'heartbeat_back_addr': '172.21.15.189:6807/2162534312', 'heartbeat_front_addr': '172.21.15.189:6805/2162534312', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.064941+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.780598+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.127700+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.730995+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.864423+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.329361+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T06:59:10.107581+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.300043+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.18', 'osds': [0, 5]}], 'pg_upmap_items': [{'pgid': '3.1f', 'mappings': [{'from': 5, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:00:07.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:00:07.940 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.739885610000002, 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.941 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:00:07.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:07.945 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.088542504999999994, 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:07.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:00:07.949 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.088527037000000003, 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.950 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:00:07.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.088515735999999998, 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:00:07.954 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:00:07.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081772262999999998, 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.959 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:07.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:07.963 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081759658999999998, 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:00:07.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:00:07.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081740143000000001, 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.968 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:00:07.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.971 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081728101999999997, 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.972 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:07.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.976 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081707799999999997, 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:00:07.980 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081695483999999999, 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.981 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:00:07.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:07.985 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081684043999999997, 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:07.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.989 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081673545, 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.990 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:00:07.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:00:07.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081663351999999995, 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.994 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:00:07.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:00:07.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081647805000000004, 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:07.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:00:08.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:00:08.002 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081618197000000003, 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.003 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081596547000000005, 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.007 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:00:08.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.011 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.081574307999999998, 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:08.015 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105530 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 661504~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829342+0000", 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.075988049000000002, 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044060136, 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.016 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105530 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829342+0000", 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829342+0000", 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829343+0000", 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829344+0000", 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0920000000000001e-06 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829347+0000", 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.163e-06 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829350+0000", 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6189999999999998e-06 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873308+0000", 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043958460999999997 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873349+0000", 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1414e-05 2024-09-14T07:00:08.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873403+0000", 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3180999999999998e-05 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.075977943000000006, 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:00:08.020 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.075967889999999996, 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.025 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:00:08.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:00:08.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:00:08.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:00:08.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.029 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.029 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.029 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:08.029 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:08.030 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:07.937+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:08.031 INFO:teuthology.orchestra.run.smithi007.stdout:2113 writesame 0xa893fa3 thru 0xa8974a9 data_size 0xf(0x3507 bytes) 2024-09-14T07:00:08.032 INFO:teuthology.orchestra.run.smithi007.stdout:2115 write 0x4a91dd5 thru 0x4aa0a6a (0xec96 bytes) 2024-09-14T07:00:08.032 INFO:teuthology.orchestra.run.smithi007.stdout:2117 punch from 0xa009123 to 0xa013d64, (0xac41 bytes) 2024-09-14T07:00:08.032 INFO:teuthology.orchestra.run.smithi007.stdout:2120 write 0x8d07fdd thru 0x8d14e01 (0xce25 bytes) 2024-09-14T07:00:08.032 INFO:teuthology.orchestra.run.smithi007.stdout:2121 read 0x355d74a thru 0x35696e7 (0xbf9e bytes) 2024-09-14T07:00:08.032 INFO:teuthology.orchestra.run.smithi007.stdout:2122 punch from 0x9317eb0 to 0x9326a40, (0xeb90 bytes) 2024-09-14T07:00:08.032 INFO:teuthology.orchestra.run.smithi007.stdout:2123 writesame 0x46155ba thru 0x462555f data_size 0x2(0xffa6 bytes) 2024-09-14T07:00:08.032 INFO:teuthology.orchestra.run.smithi007.stdout:2124 write 0x79a0254 thru 0x79adb0f (0xd8bc bytes) 2024-09-14T07:00:08.032 INFO:teuthology.orchestra.run.smithi007.stdout:2125 trunc from 0xa8974aa to 0x1cd9f55 2024-09-14T07:00:08.046 INFO:teuthology.orchestra.run.smithi007.stdout:2129 write 0x9bcd3bc thru 0x9bcea1a (0x165f bytes) 2024-09-14T07:00:08.047 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:08.046+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:08.050 INFO:teuthology.orchestra.run.smithi007.stdout:2130 write 0xe3e20e thru 0xe4b2c0 (0xd0b3 bytes) 2024-09-14T07:00:08.051 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.061 INFO:teuthology.orchestra.run.smithi007.stdout:2133 trunc from 0x9bcea1b to 0x1b433b6 2024-09-14T07:00:08.071 INFO:teuthology.orchestra.run.smithi007.stdout:2134 read 0xaca5a6 thru 0xad6e6b (0xc8c6 bytes) 2024-09-14T07:00:08.087 INFO:teuthology.orchestra.run.smithi007.stdout:2137 writesame 0xa71db42 thru 0xa72d288 data_size 0x1(0xf747 bytes) 2024-09-14T07:00:08.088 INFO:teuthology.orchestra.run.smithi007.stdout:2138 read 0x4957a0e thru 0x495e782 (0x6d75 bytes) 2024-09-14T07:00:08.092 INFO:teuthology.orchestra.run.smithi007.stdout:2139 trunc from 0xa72d289 to 0x6a26d9e 2024-09-14T07:00:08.096 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.097 INFO:teuthology.orchestra.run.smithi007.stdout:2141 punch from 0x4a3e7d9 to 0x4a408ad, (0x20d4 bytes) 2024-09-14T07:00:08.105 INFO:teuthology.orchestra.run.smithi007.stdout:2142 trunc from 0x6a26d9e to 0xe918f93 2024-09-14T07:00:08.118 INFO:teuthology.orchestra.run.smithi007.stdout:2143 trunc from 0xe918f93 to 0x7e5ad82 2024-09-14T07:00:08.123 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.123 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.134 INFO:teuthology.orchestra.run.smithi007.stdout:2146 write 0xce2d8d0 thru 0xce3bbd3 (0xe304 bytes) 2024-09-14T07:00:08.136 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:08.136 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:08.136 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:08.136 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:08.137 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.137 INFO:teuthology.orchestra.run.smithi007.stdout:2148 punch from 0x1bf66b4 to 0x1bf6ced, (0x639 bytes) 2024-09-14T07:00:08.137 INFO:teuthology.orchestra.run.smithi007.stdout:2149 write 0x3be1234 thru 0x3bef87a (0xe647 bytes) 2024-09-14T07:00:08.139 INFO:teuthology.orchestra.run.smithi007.stdout:2150 read 0x4b18a38 thru 0x4b193bf (0x988 bytes) 2024-09-14T07:00:08.140 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:08.139+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:08.145 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:08.147 INFO:teuthology.orchestra.run.smithi007.stdout:2151 trunc from 0xce3bbd4 to 0x6c43b4c 2024-09-14T07:00:08.160 INFO:teuthology.orchestra.run.smithi007.stdout:2153 trunc from 0x6c43b4c to 0xa2e9afa 2024-09-14T07:00:08.170 INFO:teuthology.orchestra.run.smithi007.stdout:2154 trunc from 0xa2e9afa to 0x64b64e6 2024-09-14T07:00:08.188 INFO:teuthology.orchestra.run.smithi007.stdout:2156 write 0xc87e2ce thru 0xc88b0cf (0xce02 bytes) 2024-09-14T07:00:08.191 INFO:teuthology.orchestra.run.smithi007.stdout:2157 write 0xa1fc398 thru 0xa204717 (0x8380 bytes) 2024-09-14T07:00:08.195 INFO:teuthology.orchestra.run.smithi007.stdout:2158 punch from 0x2cfda51 to 0x2d059f2, (0x7fa1 bytes) 2024-09-14T07:00:08.195 INFO:teuthology.orchestra.run.smithi007.stdout:2160 read 0x52a06fb thru 0x52a4703 (0x4009 bytes) 2024-09-14T07:00:08.200 INFO:teuthology.orchestra.run.smithi007.stdout:2161 trunc from 0xc88b0d0 to 0x7b65736 2024-09-14T07:00:08.209 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.210 INFO:teuthology.orchestra.run.smithi007.stdout:2163 read 0xc6d2fe thru 0xc72e7b (0x5b7e bytes) 2024-09-14T07:00:08.210 INFO:teuthology.orchestra.run.smithi007.stdout:2169 read 0x61f8f41 thru 0x61fec08 (0x5cc8 bytes) 2024-09-14T07:00:08.210 INFO:teuthology.orchestra.run.smithi007.stdout:2172 read 0x1fe0058 thru 0x1fe7cd2 (0x7c7b bytes) 2024-09-14T07:00:08.210 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.211 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.211 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.214 INFO:teuthology.orchestra.run.smithi007.stdout:2180 trunc from 0x7b65736 to 0x4102142 2024-09-14T07:00:08.221 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.221 INFO:teuthology.orchestra.run.smithi007.stdout:2182 write 0x400adc5 thru 0x400d4da (0x2716 bytes) 2024-09-14T07:00:08.224 INFO:teuthology.orchestra.run.smithi007.stdout:2183 punch from 0x9a347b to 0x9abe53, (0x89d8 bytes) 2024-09-14T07:00:08.239 INFO:teuthology.orchestra.run.smithi007.stdout:2184 writesame 0xc634608 thru 0xc6354fe data_size 0x1(0xef7 bytes) 2024-09-14T07:00:08.239 INFO:teuthology.orchestra.run.smithi007.stdout:2187 writesame 0x2fe6f06 thru 0x2ff1833 data_size 0x47(0xa92e bytes) 2024-09-14T07:00:08.240 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:08.239+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:08.242 INFO:teuthology.orchestra.run.smithi007.stdout:2188 read 0x6b45999 thru 0x6b45df9 (0x461 bytes) 2024-09-14T07:00:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:2189 read 0x9ec2c4d thru 0x9ec314e (0x502 bytes) 2024-09-14T07:00:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:2190 punch from 0xbc714c0 to 0xbc7fe34, (0xe974 bytes) 2024-09-14T07:00:08.243 INFO:teuthology.orchestra.run.smithi007.stdout:2191 write 0x96200e4 thru 0x96241ac (0x40c9 bytes) 2024-09-14T07:00:08.246 INFO:teuthology.orchestra.run.smithi007.stdout:2192 write 0x1c994eb thru 0x1ca64c3 (0xcfd9 bytes) 2024-09-14T07:00:08.248 INFO:teuthology.orchestra.run.smithi007.stdout:2193 writesame 0x912275e thru 0x91283ca data_size 0x21(0x5c6d bytes) 2024-09-14T07:00:08.249 INFO:teuthology.orchestra.run.smithi007.stdout:2195 writesame 0x43835a9 thru 0x4384aaf data_size 0x1(0x1507 bytes) 2024-09-14T07:00:08.252 INFO:teuthology.orchestra.run.smithi007.stdout:2196 write 0x11dc8d7 thru 0x11e9f88 (0xd6b2 bytes) 2024-09-14T07:00:08.253 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:08.253 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:08.253 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:08.253 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:08.253 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:08.254 INFO:teuthology.orchestra.run.smithi007.stdout:2197 read 0x79f9c79 thru 0x79f9dfa (0x182 bytes) 2024-09-14T07:00:08.254 INFO:teuthology.orchestra.run.smithi007.stdout:2198 punch from 0x85db777 to 0x85e0c5f, (0x54e8 bytes) 2024-09-14T07:00:08.254 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.254 INFO:teuthology.orchestra.run.smithi007.stdout:2200 write 0x3fa4d8a thru 0x3fb3082 (0xe2f9 bytes) 2024-09-14T07:00:08.257 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.258 INFO:teuthology.orchestra.run.smithi007.stdout:2205 write 0x8edcc78 thru 0x8eebeda (0xf263 bytes) 2024-09-14T07:00:08.259 INFO:teuthology.orchestra.run.smithi007.stdout:2206 read 0x4bf9a06 thru 0x4c06398 (0xc993 bytes) 2024-09-14T07:00:08.260 INFO:teuthology.orchestra.run.smithi007.stdout:2208 read 0x90d0919 thru 0x90d23bb (0x1aa3 bytes) 2024-09-14T07:00:08.260 INFO:teuthology.orchestra.run.smithi007.stdout:2210 read 0x2610392 thru 0x261e64c (0xe2bb bytes) 2024-09-14T07:00:08.263 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:08.268 INFO:teuthology.orchestra.run.smithi007.stdout:2212 write 0xe79d83c thru 0xe7a9749 (0xbf0e bytes) 2024-09-14T07:00:08.285 INFO:teuthology.orchestra.run.smithi007.stdout:2213 trunc from 0xe7a974a to 0x17d1892 2024-09-14T07:00:08.297 INFO:teuthology.orchestra.run.smithi007.stdout:2215 punch from 0x3cde9c to 0x3ce2a2, (0x406 bytes) 2024-09-14T07:00:08.304 INFO:teuthology.orchestra.run.smithi007.stdout:2216 write 0x26b5fe7 thru 0x26c14aa (0xb4c4 bytes) 2024-09-14T07:00:08.305 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.305 INFO:teuthology.orchestra.run.smithi007.stdout:2218 punch from 0x20f6f52 to 0x2103fc8, (0xd076 bytes) 2024-09-14T07:00:08.305 INFO:teuthology.orchestra.run.smithi007.stdout:2221 read 0x962206 thru 0x963e23 (0x1c1e bytes) 2024-09-14T07:00:08.322 INFO:teuthology.orchestra.run.smithi007.stdout:2222 write 0xc1f4fa7 thru 0xc203f20 (0xef7a bytes) 2024-09-14T07:00:08.326 INFO:teuthology.orchestra.run.smithi007.stdout:2224 write 0x4e03b85 thru 0x4e101d1 (0xc64d bytes) 2024-09-14T07:00:08.330 INFO:teuthology.orchestra.run.smithi007.stdout:2226 read 0x6da8e34 thru 0x6dafaec (0x6cb9 bytes) 2024-09-14T07:00:08.330 INFO:teuthology.orchestra.run.smithi007.stdout:2229 write 0x221d0b7 thru 0x221fdb0 (0x2cfa bytes) 2024-09-14T07:00:08.330 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.331 INFO:teuthology.orchestra.run.smithi007.stdout:2232 writesame 0x1287653 thru 0x128dafb data_size 0x1(0x64a9 bytes) 2024-09-14T07:00:08.341 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:08.340+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:08.344 INFO:teuthology.orchestra.run.smithi007.stdout:2233 punch from 0x2babfbe to 0x2bb8263, (0xc2a5 bytes) 2024-09-14T07:00:08.344 INFO:teuthology.orchestra.run.smithi007.stdout:2235 write 0x70174c2 thru 0x70231bf (0xbcfe bytes) 2024-09-14T07:00:08.350 INFO:teuthology.orchestra.run.smithi007.stdout:2236 trunc from 0xc203f21 to 0xa05a65b 2024-09-14T07:00:08.360 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.368 INFO:teuthology.orchestra.run.smithi007.stdout:2238 writesame 0xcb454c7 thru 0xcb457b0 data_size 0x1(0x2ea bytes) 2024-09-14T07:00:08.369 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.369 INFO:teuthology.orchestra.run.smithi007.stdout:2240 punch from 0x1df28f4 to 0x1df6831, (0x3f3d bytes) 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.605174517000002, 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.371 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:00:08.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:00:08.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.532055250999999, 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:08.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:00:08.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:00:08.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.531798598000002, 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.382 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:00:08.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.505511523999999, 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:00:08.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:00:08.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:08.391 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.505386909999999, 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:08.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:00:08.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.010767747999999, 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:08.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:00:08.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:08.400 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.736365087999999, 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:00:08.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:00:08.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:08.405 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.741749477999999, 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:00:08.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.551577011999999, 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.410 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:00:08.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.502443721999999, 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.419 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.134224757999998, 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:00:08.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.013655819, 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:00:08.424 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:00:08.428 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.882739938, 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.429 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:00:08.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.433 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.853702676000001, 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:00:08.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:00:08.438 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.827933254000001, 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:00:08.442 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.794575111, 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:00:08.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.630367329999999, 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:00:08.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.61898807, 2024-09-14T07:00:08.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:08.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:00:08.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:00:08.456 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.618940376000001, 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:00:08.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:00:08.461 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 23.618598050999999, 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:00:08.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:00:08.466 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.466 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.466 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.466 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.466 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.466 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:08.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:00:08.468 INFO:teuthology.orchestra.run.smithi007.stdout:2241 trunc from 0xcb457b1 to 0x883f259 2024-09-14T07:00:08.468 INFO:teuthology.orchestra.run.smithi007.stdout:2242 writesame 0x2639af7 thru 0x2644a51 data_size 0x4d(0xaf5b bytes) 2024-09-14T07:00:08.468 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.468 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.468 INFO:teuthology.orchestra.run.smithi007.stdout:2245 read 0x2f7353c thru 0x2f74b10 (0x15d5 bytes) 2024-09-14T07:00:08.468 INFO:teuthology.orchestra.run.smithi007.stdout:2246 writesame 0x3c911b1 thru 0x3c9c098 data_size 0x3a(0xaee8 bytes) 2024-09-14T07:00:08.470 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:08.470 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:00:08.471 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:00:08.472 INFO:teuthology.orchestra.run.smithi007.stdout:2247 trunc from 0x883f259 to 0xdaaf7ec 2024-09-14T07:00:08.472 INFO:teuthology.orchestra.run.smithi007.stdout:2248 write 0xa5d555f thru 0xa5dc247 (0x6ce9 bytes) 2024-09-14T07:00:08.472 INFO:teuthology.orchestra.run.smithi007.stdout:2249 write 0x7510263 thru 0x751df95 (0xdd33 bytes) 2024-09-14T07:00:08.472 INFO:teuthology.orchestra.run.smithi007.stdout:2250 write 0x301d94c thru 0x3022a4a (0x50ff bytes) 2024-09-14T07:00:08.472 INFO:teuthology.orchestra.run.smithi007.stdout:2251 writesame 0x415ed53 thru 0x416d00d data_size 0x1(0xe2bb bytes) 2024-09-14T07:00:08.473 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-14T07:00:07.758225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599692918,"num_objects":1157,"num_object_clones":264,"num_object_copies":2314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1157,"num_whiteouts":642,"num_read":120710,"num_read_kb":298925,"num_write":6250,"num_write_kb":152990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":9336079,"num_keys_recovered":59,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5746,"ondisk_log_size":5746,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":271416,"kb_used_data":23916,"kb_used_omap":10,"kb_used_meta":247285,"kb_avail":660331464,"statfs":{"total":676457349120,"available":676179419136,"internally_reserved":0,"allocated":24489984,"data_stored":180163564,"data_compressed":5821606,"data_compressed_allocated":14958592,"data_compressed_original":175266003,"omap_allocated":11157,"internal_metadata":253219947},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":110178324,"num_objects":194,"num_object_clones":45,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":138,"num_read":24259,"num_read_kb":54785,"num_write":899,"num_write_kb":22490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.367145"},"pg_stats":[{"pgid":"3.1f","version":"54'168","reported_seq":6118,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.481610+0000","last_change":"2024-09-14T07:00:01.385727+0000","last_active":"2024-09-14T07:00:07.481610+0000","last_peered":"2024-09-14T07:00:07.481610+0000","last_clean":"2024-09-14T07:00:07.481610+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:00:07.481610+0000","last_undegraded":"2024-09-14T07:00:07.481610+0000","last_fullsized":"2024-09-14T07:00:07.481610+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:45:44.647225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043615199999999998,"stat_sum":{"num_bytes":36013870,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":18,"num_read":6753,"num_read_kb":23216,"num_write":152,"num_write_kb":14449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"54'105","reported_seq":253,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:00:05.469946+0000","last_change":"2024-09-14T06:59:30.808971+0000","last_active":"2024-09-14T07:00:05.469946+0000","last_peered":"2024-09-14T07:00:05.469946+0000","last_clean":"2024-09-14T07:00:05.469946+0000","last_became_active":"2024-09-14T06:59:28.801728+0000","last_became_peered":"2024-09-14T06:59:28.801728+0000","last_unstale":"2024-09-14T07:00:05.469946+0000","last_undegraded":"2024-09-14T07:00:05.469946+0000","last_fullsized":"2024-09-14T07:00:05.469946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:34:30.150964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000412824,"stat_sum":{"num_bytes":9798480,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":23,"num_read":106,"num_read_kb":1098,"num_write":92,"num_write_kb":2094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"55'142","reported_seq":4411,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.412182+0000","last_change":"2024-09-14T06:59:30.803414+0000","last_active":"2024-09-14T07:00:07.412182+0000","last_peered":"2024-09-14T07:00:07.412182+0000","last_clean":"2024-09-14T07:00:07.412182+0000","last_became_active":"2024-09-14T06:59:28.799714+0000","last_became_peered":"2024-09-14T06:59:28.799714+0000","last_unstale":"2024-09-14T07:00:07.412182+0000","last_undegraded":"2024-09-14T07:00:07.412182+0000","last_fullsized":"2024-09-14T07:00:07.412182+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:07:34.851566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001689915,"stat_sum":{"num_bytes":7776422,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":25,"num_read":4430,"num_read_kb":9919,"num_write":147,"num_write_kb":4337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"56'101","reported_seq":679,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.514297+0000","last_change":"2024-09-14T06:59:30.806817+0000","last_active":"2024-09-14T07:00:07.514297+0000","last_peered":"2024-09-14T07:00:07.514297+0000","last_clean":"2024-09-14T07:00:07.514297+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:00:07.514297+0000","last_undegraded":"2024-09-14T07:00:07.514297+0000","last_fullsized":"2024-09-14T07:00:07.514297+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:09:31.914639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042050300000000002,"stat_sum":{"num_bytes":3971872,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":20,"num_read":555,"num_read_kb":2256,"num_write":93,"num_write_kb":2218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"54'184","reported_seq":857,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-14T07:00:06.409652+0000","last_change":"2024-09-14T06:59:30.801680+0000","last_active":"2024-09-14T07:00:06.409652+0000","last_peered":"2024-09-14T07:00:06.409652+0000","last_clean":"2024-09-14T07:00:06.409652+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:00:06.409652+0000","last_undegraded":"2024-09-14T07:00:06.409652+0000","last_fullsized":"2024-09-14T07:00:06.409652+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:33:35.619140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060885900000000001,"stat_sum":{"num_bytes":29238751,"num_objects":42,"num_object_clones":10,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":903,"num_read_kb":8016,"num_write":203,"num_write_kb":5892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"56'135","reported_seq":3873,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.522567+0000","last_change":"2024-09-14T06:59:30.807597+0000","last_active":"2024-09-14T07:00:07.522567+0000","last_peered":"2024-09-14T07:00:07.522567+0000","last_clean":"2024-09-14T07:00:07.522567+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:00:07.522567+0000","last_undegraded":"2024-09-14T07:00:07.522567+0000","last_fullsized":"2024-09-14T07:00:07.522567+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:32.605284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010420550000000001,"stat_sum":{"num_bytes":18749257,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":19,"num_read":3881,"num_read_kb":6466,"num_write":139,"num_write_kb":2174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"55'248","reported_seq":3424,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-14T07:00:06.457303+0000","last_change":"2024-09-14T06:59:30.801946+0000","last_active":"2024-09-14T07:00:06.457303+0000","last_peered":"2024-09-14T07:00:06.457303+0000","last_clean":"2024-09-14T07:00:06.457303+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:00:06.457303+0000","last_undegraded":"2024-09-14T07:00:06.457303+0000","last_fullsized":"2024-09-14T07:00:06.457303+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:40.316479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081998900000000003,"stat_sum":{"num_bytes":63545105,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":23,"num_read":3692,"num_read_kb":9714,"num_write":291,"num_write_kb":10071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"54'106","reported_seq":892,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.464863+0000","last_change":"2024-09-14T06:59:37.046075+0000","last_active":"2024-09-14T07:00:07.464863+0000","last_peered":"2024-09-14T07:00:07.464863+0000","last_clean":"2024-09-14T07:00:07.464863+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:00:07.464863+0000","last_undegraded":"2024-09-14T07:00:07.464863+0000","last_fullsized":"2024-09-14T07:00:07.464863+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:21.940258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059456000000000005,"stat_sum":{"num_bytes":7980957,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":22,"num_read":1156,"num_read_kb":3752,"num_write":80,"num_write_kb":2431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"54'96","reported_seq":2986,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:00:05.466158+0000","last_change":"2024-09-14T06:59:30.809133+0000","last_active":"2024-09-14T07:00:05.466158+0000","last_peered":"2024-09-14T07:00:05.466158+0000","last_clean":"2024-09-14T07:00:05.466158+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:00:05.466158+0000","last_undegraded":"2024-09-14T07:00:05.466158+0000","last_fullsized":"2024-09-14T07:00:05.466158+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:52:58.009748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055309100000000002,"stat_sum":{"num_bytes":5993453,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":24,"num_read":3566,"num_read_kb":8681,"num_write":66,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"52'66","reported_seq":2748,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.411343+0000","last_change":"2024-09-14T06:59:30.803754+0000","last_active":"2024-09-14T07:00:07.411343+0000","last_peered":"2024-09-14T07:00:07.411343+0000","last_clean":"2024-09-14T07:00:07.411343+0000","last_became_active":"2024-09-14T06:59:28.801458+0000","last_became_peered":"2024-09-14T06:59:28.801458+0000","last_unstale":"2024-09-14T07:00:07.411343+0000","last_undegraded":"2024-09-14T07:00:07.411343+0000","last_fullsized":"2024-09-14T07:00:07.411343+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:27:30.867439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086036299999999995,"stat_sum":{"num_bytes":5128708,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":2610,"num_read_kb":5692,"num_write":56,"num_write_kb":2143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"54'115","reported_seq":459,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.461857+0000","last_change":"2024-09-14T06:59:51.355900+0000","last_active":"2024-09-14T07:00:07.461857+0000","last_peered":"2024-09-14T07:00:07.461857+0000","last_clean":"2024-09-14T07:00:07.461857+0000","last_became_active":"2024-09-14T06:59:51.330515+0000","last_became_peered":"2024-09-14T06:59:51.330515+0000","last_unstale":"2024-09-14T07:00:07.461857+0000","last_undegraded":"2024-09-14T07:00:07.461857+0000","last_fullsized":"2024-09-14T07:00:07.461857+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:36:22.600056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052297400000000005,"stat_sum":{"num_bytes":6307612,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":27,"num_read":246,"num_read_kb":1227,"num_write":91,"num_write_kb":2204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"56'501","reported_seq":3991,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.549736+0000","last_change":"2024-09-14T06:59:30.802873+0000","last_active":"2024-09-14T07:00:07.549736+0000","last_peered":"2024-09-14T07:00:07.549736+0000","last_clean":"2024-09-14T07:00:07.549736+0000","last_became_active":"2024-09-14T06:59:28.801310+0000","last_became_peered":"2024-09-14T06:59:28.801310+0000","last_unstale":"2024-09-14T07:00:07.549736+0000","last_undegraded":"2024-09-14T07:00:07.549736+0000","last_fullsized":"2024-09-14T07:00:07.549736+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":501,"log_dups_size":0,"ondisk_log_size":501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:26:28.006901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013224059999999999,"stat_sum":{"num_bytes":15490593,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":21,"num_read":5998,"num_read_kb":8657,"num_write":684,"num_write_kb":3342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":88,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.412446+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:00:07.412446+0000","last_peered":"2024-09-14T07:00:07.412446+0000","last_clean":"2024-09-14T07:00:07.412446+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:00:07.412446+0000","last_undegraded":"2024-09-14T07:00:07.412446+0000","last_fullsized":"2024-09-14T07:00:07.412446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"55'203","reported_seq":599,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-14T07:00:06.504307+0000","last_change":"2024-09-14T06:59:30.801713+0000","last_active":"2024-09-14T07:00:06.504307+0000","last_peered":"2024-09-14T07:00:06.504307+0000","last_clean":"2024-09-14T07:00:06.504307+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:00:06.504307+0000","last_undegraded":"2024-09-14T07:00:06.504307+0000","last_fullsized":"2024-09-14T07:00:06.504307+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:24:21.270175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059990300000000005,"stat_sum":{"num_bytes":32889500,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":18,"num_read":1124,"num_read_kb":1417,"num_write":198,"num_write_kb":3425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":86,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-14T07:00:06.409722+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:00:06.409722+0000","last_peered":"2024-09-14T07:00:06.409722+0000","last_clean":"2024-09-14T07:00:06.409722+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:00:06.409722+0000","last_undegraded":"2024-09-14T07:00:06.409722+0000","last_fullsized":"2024-09-14T07:00:06.409722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"52'121","reported_seq":4185,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-14T07:00:06.409701+0000","last_change":"2024-09-14T06:59:51.791781+0000","last_active":"2024-09-14T07:00:06.409701+0000","last_peered":"2024-09-14T07:00:06.409701+0000","last_clean":"2024-09-14T07:00:06.409701+0000","last_became_active":"2024-09-14T06:59:51.751790+0000","last_became_peered":"2024-09-14T06:59:51.751790+0000","last_unstale":"2024-09-14T07:00:06.409701+0000","last_undegraded":"2024-09-14T07:00:06.409701+0000","last_fullsized":"2024-09-14T07:00:06.409701+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:46.840399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060862000000000004,"stat_sum":{"num_bytes":5015210,"num_objects":29,"num_object_clones":5,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":16,"num_read":5143,"num_read_kb":9372,"num_write":149,"num_write_kb":3427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":88,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.412325+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:00:07.412325+0000","last_peered":"2024-09-14T07:00:07.412325+0000","last_clean":"2024-09-14T07:00:07.412325+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:00:07.412325+0000","last_undegraded":"2024-09-14T07:00:07.412325+0000","last_fullsized":"2024-09-14T07:00:07.412325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"54'100","reported_seq":997,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:00:05.466318+0000","last_change":"2024-09-14T06:59:51.343911+0000","last_active":"2024-09-14T07:00:05.466318+0000","last_peered":"2024-09-14T07:00:05.466318+0000","last_clean":"2024-09-14T07:00:05.466318+0000","last_became_active":"2024-09-14T06:59:51.331353+0000","last_became_peered":"2024-09-14T06:59:51.331353+0000","last_unstale":"2024-09-14T07:00:05.466318+0000","last_undegraded":"2024-09-14T07:00:05.466318+0000","last_fullsized":"2024-09-14T07:00:05.466318+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:49:13.202362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2712106,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":17,"num_read":915,"num_read_kb":670,"num_write":71,"num_write_kb":526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":3893,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":90,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:00:04.425262+0000","last_change":"2024-09-14T06:59:51.331407+0000","last_active":"2024-09-14T07:00:04.425262+0000","last_peered":"2024-09-14T07:00:04.425262+0000","last_clean":"2024-09-14T07:00:04.425262+0000","last_became_active":"2024-09-14T06:59:51.331192+0000","last_became_peered":"2024-09-14T06:59:51.331192+0000","last_unstale":"2024-09-14T07:00:04.425262+0000","last_undegraded":"2024-09-14T07:00:04.425262+0000","last_fullsized":"2024-09-14T07:00:04.425262+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:41:42.693471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"56'331","reported_seq":4302,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.535280+0000","last_change":"2024-09-14T06:59:51.387715+0000","last_active":"2024-09-14T07:00:07.535280+0000","last_peered":"2024-09-14T07:00:07.535280+0000","last_clean":"2024-09-14T07:00:07.535280+0000","last_became_active":"2024-09-14T06:59:51.336748+0000","last_became_peered":"2024-09-14T06:59:51.336748+0000","last_unstale":"2024-09-14T07:00:07.535280+0000","last_undegraded":"2024-09-14T07:00:07.535280+0000","last_fullsized":"2024-09-14T07:00:07.535280+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:46:56.708403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052993500000000004,"stat_sum":{"num_bytes":25156437,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":23,"num_read":2890,"num_read_kb":5504,"num_write":329,"num_write_kb":5092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":1805025,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"54'244","reported_seq":481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.461838+0000","last_change":"2024-09-14T06:59:30.810757+0000","last_active":"2024-09-14T07:00:07.461838+0000","last_peered":"2024-09-14T07:00:07.461838+0000","last_clean":"2024-09-14T07:00:07.461838+0000","last_became_active":"2024-09-14T06:59:28.802025+0000","last_became_peered":"2024-09-14T06:59:28.802025+0000","last_unstale":"2024-09-14T07:00:07.461838+0000","last_undegraded":"2024-09-14T07:00:07.461838+0000","last_fullsized":"2024-09-14T07:00:07.461838+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:11.846297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044476100000000001,"stat_sum":{"num_bytes":13422750,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":22,"num_read":974,"num_read_kb":1213,"num_write":359,"num_write_kb":4053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.411906+0000","last_change":"2024-09-14T06:59:51.752635+0000","last_active":"2024-09-14T07:00:07.411906+0000","last_peered":"2024-09-14T07:00:07.411906+0000","last_clean":"2024-09-14T07:00:07.411906+0000","last_became_active":"2024-09-14T06:59:51.749953+0000","last_became_peered":"2024-09-14T06:59:51.749953+0000","last_unstale":"2024-09-14T07:00:07.411906+0000","last_undegraded":"2024-09-14T07:00:07.411906+0000","last_fullsized":"2024-09-14T07:00:07.411906+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:31:47.116689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"54'100","reported_seq":16208,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:00:05.523317+0000","last_change":"2024-09-14T06:59:30.809288+0000","last_active":"2024-09-14T07:00:05.523317+0000","last_peered":"2024-09-14T07:00:05.523317+0000","last_clean":"2024-09-14T07:00:05.523317+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:00:05.523317+0000","last_undegraded":"2024-09-14T07:00:05.523317+0000","last_fullsized":"2024-09-14T07:00:05.523317+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:36:18.612890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063151899999999998,"stat_sum":{"num_bytes":12476423,"num_objects":31,"num_object_clones":9,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":16074,"num_read_kb":34181,"num_write":150,"num_write_kb":15799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":83,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:00:04.512232+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:00:04.512232+0000","last_peered":"2024-09-14T07:00:04.512232+0000","last_clean":"2024-09-14T07:00:04.512232+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:00:04.512232+0000","last_undegraded":"2024-09-14T07:00:04.512232+0000","last_fullsized":"2024-09-14T07:00:04.512232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"52'117","reported_seq":1179,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.500633+0000","last_change":"2024-09-14T06:59:51.403760+0000","last_active":"2024-09-14T07:00:07.500633+0000","last_peered":"2024-09-14T07:00:07.500633+0000","last_clean":"2024-09-14T07:00:07.500633+0000","last_became_active":"2024-09-14T06:59:51.338478+0000","last_became_peered":"2024-09-14T06:59:51.338478+0000","last_unstale":"2024-09-14T07:00:07.500633+0000","last_undegraded":"2024-09-14T07:00:07.500633+0000","last_fullsized":"2024-09-14T07:00:07.500633+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:28:07.881594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3617805,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":23,"num_read":2014,"num_read_kb":2657,"num_write":79,"num_write_kb":663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":270336,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":88,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.410747+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T07:00:07.410747+0000","last_peered":"2024-09-14T07:00:07.410747+0000","last_clean":"2024-09-14T07:00:07.410747+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:00:07.410747+0000","last_undegraded":"2024-09-14T07:00:07.410747+0000","last_fullsized":"2024-09-14T07:00:07.410747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"54'314","reported_seq":7831,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-14T07:00:06.409640+0000","last_change":"2024-09-14T06:59:30.801433+0000","last_active":"2024-09-14T07:00:06.409640+0000","last_peered":"2024-09-14T07:00:06.409640+0000","last_clean":"2024-09-14T07:00:06.409640+0000","last_became_active":"2024-09-14T06:59:28.801659+0000","last_became_peered":"2024-09-14T06:59:28.801659+0000","last_unstale":"2024-09-14T07:00:06.409640+0000","last_undegraded":"2024-09-14T07:00:06.409640+0000","last_fullsized":"2024-09-14T07:00:06.409640+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":314,"log_dups_size":0,"ondisk_log_size":314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:05:55.332255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037135599999999999,"stat_sum":{"num_bytes":40575657,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":21,"num_read":8617,"num_read_kb":31402,"num_write":481,"num_write_kb":10636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":89,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.411748+0000","last_change":"2024-09-14T06:59:21.767446+0000","last_active":"2024-09-14T07:00:07.411748+0000","last_peered":"2024-09-14T07:00:07.411748+0000","last_clean":"2024-09-14T07:00:07.411748+0000","last_became_active":"2024-09-14T06:59:20.120193+0000","last_became_peered":"2024-09-14T06:59:20.120193+0000","last_unstale":"2024-09-14T07:00:07.411748+0000","last_undegraded":"2024-09-14T07:00:07.411748+0000","last_fullsized":"2024-09-14T07:00:07.411748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:18.122142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":178,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.425869+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T07:00:07.425869+0000","last_peered":"2024-09-14T07:00:07.425869+0000","last_clean":"2024-09-14T07:00:07.425869+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:00:07.425869+0000","last_undegraded":"2024-09-14T07:00:07.425869+0000","last_fullsized":"2024-09-14T07:00:07.425869+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"55'183","reported_seq":6158,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-14T07:00:06.510400+0000","last_change":"2024-09-14T06:59:30.801426+0000","last_active":"2024-09-14T07:00:06.510400+0000","last_peered":"2024-09-14T07:00:06.510400+0000","last_clean":"2024-09-14T07:00:06.510400+0000","last_became_active":"2024-09-14T06:59:28.808414+0000","last_became_peered":"2024-09-14T06:59:28.808414+0000","last_unstale":"2024-09-14T07:00:06.510400+0000","last_undegraded":"2024-09-14T07:00:06.510400+0000","last_fullsized":"2024-09-14T07:00:06.510400+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:10:45.347235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038565499999999999,"stat_sum":{"num_bytes":19818549,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":6197,"num_read_kb":14183,"num_write":200,"num_write_kb":7610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"55'222","reported_seq":3324,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.475911+0000","last_change":"2024-09-14T06:59:30.802594+0000","last_active":"2024-09-14T07:00:07.475911+0000","last_peered":"2024-09-14T07:00:07.475911+0000","last_clean":"2024-09-14T07:00:07.475911+0000","last_became_active":"2024-09-14T06:59:28.804973+0000","last_became_peered":"2024-09-14T06:59:28.804973+0000","last_unstale":"2024-09-14T07:00:07.475911+0000","last_undegraded":"2024-09-14T07:00:07.475911+0000","last_fullsized":"2024-09-14T07:00:07.475911+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:07:58.379209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793000000000001,"stat_sum":{"num_bytes":4313088,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":16,"num_read":5312,"num_read_kb":6961,"num_write":192,"num_write_kb":1609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"56'251","reported_seq":1608,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.512183+0000","last_change":"2024-09-14T06:59:30.802721+0000","last_active":"2024-09-14T07:00:07.512183+0000","last_peered":"2024-09-14T07:00:07.512183+0000","last_clean":"2024-09-14T07:00:07.512183+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:00:07.512183+0000","last_undegraded":"2024-09-14T07:00:07.512183+0000","last_fullsized":"2024-09-14T07:00:07.512183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:29:35.542770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402691,"stat_sum":{"num_bytes":3008996,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":2053,"num_read_kb":2458,"num_write":339,"num_write_kb":1463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"52'108","reported_seq":1581,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-14T07:00:04.512426+0000","last_change":"2024-09-14T06:59:51.385585+0000","last_active":"2024-09-14T07:00:04.512426+0000","last_peered":"2024-09-14T07:00:04.512426+0000","last_clean":"2024-09-14T07:00:04.512426+0000","last_became_active":"2024-09-14T06:59:51.333054+0000","last_became_peered":"2024-09-14T06:59:51.333054+0000","last_unstale":"2024-09-14T07:00:04.512426+0000","last_undegraded":"2024-09-14T07:00:04.512426+0000","last_fullsized":"2024-09-14T07:00:04.512426+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:36.429393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056983800000000003,"stat_sum":{"num_bytes":34314884,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":19,"num_read":1268,"num_read_kb":5991,"num_write":86,"num_write_kb":1503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":522922,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"55'158","reported_seq":2621,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.461862+0000","last_change":"2024-09-14T06:59:30.811244+0000","last_active":"2024-09-14T07:00:07.461862+0000","last_peered":"2024-09-14T07:00:07.461862+0000","last_clean":"2024-09-14T07:00:07.461862+0000","last_became_active":"2024-09-14T06:59:28.799619+0000","last_became_peered":"2024-09-14T06:59:28.799619+0000","last_unstale":"2024-09-14T07:00:07.461862+0000","last_undegraded":"2024-09-14T07:00:07.461862+0000","last_fullsized":"2024-09-14T07:00:07.461862+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:36:17.712415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077164499999999995,"stat_sum":{"num_bytes":26900260,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":19,"num_read":2531,"num_read_kb":6470,"num_write":151,"num_write_kb":6275,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"54'179","reported_seq":445,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.502189+0000","last_change":"2024-09-14T06:59:30.803608+0000","last_active":"2024-09-14T07:00:07.502189+0000","last_peered":"2024-09-14T07:00:07.502189+0000","last_clean":"2024-09-14T07:00:07.502189+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:00:07.502189+0000","last_undegraded":"2024-09-14T07:00:07.502189+0000","last_fullsized":"2024-09-14T07:00:07.502189+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:18:09.864166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016847640000000001,"stat_sum":{"num_bytes":14496252,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":21,"num_read":640,"num_read_kb":2959,"num_write":230,"num_write_kb":4725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"54'132","reported_seq":10481,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.411431+0000","last_change":"2024-09-14T06:59:30.803975+0000","last_active":"2024-09-14T07:00:07.411431+0000","last_peered":"2024-09-14T07:00:07.411431+0000","last_clean":"2024-09-14T07:00:07.411431+0000","last_became_active":"2024-09-14T06:59:28.801111+0000","last_became_peered":"2024-09-14T06:59:28.801111+0000","last_unstale":"2024-09-14T07:00:07.411431+0000","last_undegraded":"2024-09-14T07:00:07.411431+0000","last_fullsized":"2024-09-14T07:00:07.411431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:26:56.703145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084307800000000003,"stat_sum":{"num_bytes":44164474,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":23,"num_read":10592,"num_read_kb":41192,"num_write":124,"num_write_kb":12071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"54'89","reported_seq":2650,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.462072+0000","last_change":"2024-09-14T06:59:30.811224+0000","last_active":"2024-09-14T07:00:07.462072+0000","last_peered":"2024-09-14T07:00:07.462072+0000","last_clean":"2024-09-14T07:00:07.462072+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:00:07.462072+0000","last_undegraded":"2024-09-14T07:00:07.462072+0000","last_fullsized":"2024-09-14T07:00:07.462072+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:21:52.974099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053083799999999995,"stat_sum":{"num_bytes":8050225,"num_objects":46,"num_object_clones":9,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":2516,"num_read_kb":6936,"num_write":69,"num_write_kb":3037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"54'162","reported_seq":2583,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.426038+0000","last_change":"2024-09-14T06:59:30.806812+0000","last_active":"2024-09-14T07:00:07.426038+0000","last_peered":"2024-09-14T07:00:07.426038+0000","last_clean":"2024-09-14T07:00:07.426038+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:00:07.426038+0000","last_undegraded":"2024-09-14T07:00:07.426038+0000","last_fullsized":"2024-09-14T07:00:07.426038+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:07:18.169370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055264400000000003,"stat_sum":{"num_bytes":10631909,"num_objects":27,"num_object_clones":7,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":13,"num_read":2689,"num_read_kb":8706,"num_write":176,"num_write_kb":4398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"54'272","reported_seq":8930,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.470415+0000","last_change":"2024-09-14T06:59:30.802301+0000","last_active":"2024-09-14T07:00:07.470415+0000","last_peered":"2024-09-14T07:00:07.470415+0000","last_clean":"2024-09-14T07:00:07.470415+0000","last_became_active":"2024-09-14T06:59:28.801454+0000","last_became_peered":"2024-09-14T06:59:28.801454+0000","last_unstale":"2024-09-14T07:00:07.470415+0000","last_undegraded":"2024-09-14T07:00:07.470415+0000","last_fullsized":"2024-09-14T07:00:07.470415+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:57:20.012918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010054669999999999,"stat_sum":{"num_bytes":13662457,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":19,"num_read":10106,"num_read_kb":21152,"num_write":362,"num_write_kb":6637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"54'182","reported_seq":950,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.412169+0000","last_change":"2024-09-14T06:59:54.343771+0000","last_active":"2024-09-14T07:00:07.412169+0000","last_peered":"2024-09-14T07:00:07.412169+0000","last_clean":"2024-09-14T07:00:07.412169+0000","last_became_active":"2024-09-14T06:59:54.343180+0000","last_became_peered":"2024-09-14T06:59:54.343180+0000","last_unstale":"2024-09-14T07:00:07.412169+0000","last_undegraded":"2024-09-14T07:00:07.412169+0000","last_fullsized":"2024-09-14T07:00:07.412169+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:36.793910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000465725,"stat_sum":{"num_bytes":31181143,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":18,"num_read":1239,"num_read_kb":2557,"num_write":145,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":292050,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"56'236","reported_seq":3770,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-14T07:00:07.444556+0000","last_change":"2024-09-14T06:59:30.802121+0000","last_active":"2024-09-14T07:00:07.444556+0000","last_peered":"2024-09-14T07:00:07.444556+0000","last_clean":"2024-09-14T07:00:07.444556+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:00:07.444556+0000","last_undegraded":"2024-09-14T07:00:07.444556+0000","last_fullsized":"2024-09-14T07:00:07.444556+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:51.662558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087321199999999995,"stat_sum":{"num_bytes":42699326,"num_objects":43,"num_object_clones":14,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":20,"num_read":3854,"num_read_kb":4195,"num_write":163,"num_write_kb":3427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":599102531,"num_objects":1153,"num_object_clones":264,"num_object_copies":2306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1153,"num_whiteouts":642,"num_read":120644,"num_read_kb":298870,"num_write":6147,"num_write_kb":151360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":9336060,"num_keys_recovered":59,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19525632,"data_stored":173826945,"data_compressed":5535568,"data_compressed_allocated":14430208,"data_compressed_original":170883810,"omap_allocated":0,"internal_metadata":0},"log_size":5671,"ondisk_log_size":5671,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":0,"seq":64424509455,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43012,"kb_used_data":3504,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328828,"statfs":{"total":96636764160,"available":96592719872,"internally_reserved":0,"allocated":3588096,"data_stored":25560790,"data_compressed":865018,"data_compressed_allocated":2138112,"data_compressed_original":24794917,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43100,"kb_used_data":3552,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328740,"statfs":{"total":96636764160,"available":96592629760,"internally_reserved":0,"allocated":3637248,"data_stored":23039213,"data_compressed":892734,"data_compressed_allocated":2220032,"data_compressed_original":22295802,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38960,"kb_used_data":3612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332880,"statfs":{"total":96636764160,"available":96596869120,"internally_reserved":0,"allocated":3698688,"data_stored":31576918,"data_compressed":840572,"data_compressed_allocated":2392064,"data_compressed_original":30922442,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39960,"kb_used_data":4612,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331880,"statfs":{"total":96636764160,"available":96595845120,"internally_reserved":0,"allocated":4722688,"data_stored":33269853,"data_compressed":1295207,"data_compressed_allocated":2945024,"data_compressed_original":32294602,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33756,"kb_used_data":2568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338084,"statfs":{"total":96636764160,"available":96602198016,"internally_reserved":0,"allocated":2629632,"data_stored":23610370,"data_compressed":520575,"data_compressed_allocated":1593344,"data_compressed_original":23111651,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542158,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38932,"kb_used_data":3560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332908,"statfs":{"total":96636764160,"available":96596897792,"internally_reserved":0,"allocated":3645440,"data_stored":24227627,"data_compressed":909752,"data_compressed_allocated":2273280,"data_compressed_original":23552348,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33696,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338144,"statfs":{"total":96636764160,"available":96602259456,"internally_reserved":0,"allocated":2568192,"data_stored":18878793,"data_compressed":497748,"data_compressed_allocated":1396736,"data_compressed_original":18294241,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3891200,"data_stored":31535056,"data_compressed":1199702,"data_compressed_allocated":2768896,"data_compressed_original":30848714,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2994176,"data_stored":30229882,"data_compressed":834705,"data_compressed_allocated":2326528,"data_compressed_original":29865674,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2768896,"data_stored":21796249,"data_compressed":774731,"data_compressed_allocated":2048000,"data_compressed_original":21366537,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2854912,"data_stored":24731959,"data_compressed":828765,"data_compressed_allocated":2056192,"data_compressed_original":24254245,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2908160,"data_stored":23858210,"data_compressed":835540,"data_compressed_allocated":2195456,"data_compressed_original":23474524,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1957888,"data_stored":22750157,"data_compressed":515444,"data_compressed_allocated":1556480,"data_compressed_original":22538211,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":310283,"data_compressed":48933,"data_compressed_allocated":81920,"data_compressed_original":241664,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1966080,"data_stored":18615149,"data_compressed":497748,"data_compressed_allocated":1396736,"data_compressed_original":18294241,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:00:08.475 INFO:tasks.thrashosds.thrasher:Setting 3.b to [7, 5, 6, 4] 2024-09-14T07:00:08.475 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.b', '7', '5', '6', '4'] 2024-09-14T07:00:08.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.b 7 5 6 4 2024-09-14T07:00:08.477 INFO:teuthology.orchestra.run.smithi007.stdout:2252 writesame 0xbb948a2 thru 0xbb99422 data_size 0x11(0x4b81 bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2255 write 0x21c69ff thru 0x21c72af (0x8b1 bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2258 read 0x813146c thru 0x8139a5f (0x85f4 bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2260 write 0x6778e28 thru 0x677ccc0 (0x3e99 bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2261 read 0x636ac07 thru 0x637634d (0xb747 bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2263 punch from 0x88d61e5 to 0x88e3e67, (0xdc82 bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2266 write 0x59c7eab thru 0x59cae7a (0x2fd0 bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2267 punch from 0xb3b2bb3 to 0xb3c0a10, (0xde5d bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2268 write 0x450d0b6 thru 0x4516098 (0x8fe3 bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2269 write 0x7769028 thru 0x776e2db (0x52b4 bytes) 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2270 trunc from 0xdaaf7ec to 0xaa3b667 2024-09-14T07:00:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:2271 writesame 0x9be0191 thru 0x9be64c1 data_size 0x1(0x6331 bytes) 2024-09-14T07:00:08.479 INFO:teuthology.orchestra.run.smithi007.stdout:2272 read 0x92b8b2b thru 0x92c406f (0xb545 bytes) 2024-09-14T07:00:08.479 INFO:teuthology.orchestra.run.smithi007.stdout:2273 punch from 0x9b501de to 0x9b597aa, (0x95cc bytes) 2024-09-14T07:00:08.479 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.479 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.479 INFO:teuthology.orchestra.run.smithi007.stdout:2276 write 0x6beb151 thru 0x6bf0ebc (0x5d6c bytes) 2024-09-14T07:00:08.479 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:08.467+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:08.480 INFO:teuthology.orchestra.run.smithi007.stdout:2277 read 0x3efd927 thru 0x3f001ca (0x28a4 bytes) 2024-09-14T07:00:08.480 INFO:teuthology.orchestra.run.smithi007.stdout:2279 trunc from 0xaa3b667 to 0xc655f03 2024-09-14T07:00:08.480 INFO:teuthology.orchestra.run.smithi007.stdout:2280 write 0x4942e3d thru 0x4943102 (0x2c6 bytes) 2024-09-14T07:00:08.480 INFO:teuthology.orchestra.run.smithi007.stdout:2282 write 0x8bff2a thru 0x8cd43b (0xd512 bytes) 2024-09-14T07:00:08.481 INFO:teuthology.orchestra.run.smithi007.stdout:2283 trunc from 0xc655f03 to 0xd2c6741 2024-09-14T07:00:08.486 INFO:teuthology.orchestra.run.smithi007.stdout:2284 read 0xb285c3 thru 0xb34dbd (0xc7fb bytes) 2024-09-14T07:00:08.487 INFO:teuthology.orchestra.run.smithi007.stdout:2285 read 0x1b3765b thru 0x1b42b23 (0xb4c9 bytes) 2024-09-14T07:00:08.487 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.488 INFO:teuthology.orchestra.run.smithi007.stdout:2288 flatten 2024-09-14T07:00:08.503 INFO:teuthology.orchestra.run.smithi007.stdout:2289 write 0xd56f9ce thru 0xd572a73 (0x30a6 bytes) 2024-09-14T07:00:08.512 INFO:teuthology.orchestra.run.smithi007.stdout:2290 trunc from 0xd572a74 to 0x7d510ef 2024-09-14T07:00:08.536 INFO:teuthology.orchestra.run.smithi007.stdout:2291 write 0xdbc090b thru 0xdbc8533 (0x7c29 bytes) 2024-09-14T07:00:08.539 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.539 INFO:teuthology.orchestra.run.smithi007.stdout:2293 write 0x5c5c3f thru 0x5c741f (0x17e1 bytes) 2024-09-14T07:00:08.541 INFO:teuthology.orchestra.run.smithi007.stdout:2294 writesame 0x1ade787 thru 0x1ae4e9b data_size 0x1(0x6715 bytes) 2024-09-14T07:00:08.553 INFO:teuthology.orchestra.run.smithi007.stdout:2295 writesame 0x3e1dd43 thru 0x3e1e3dd data_size 0x1(0x69b bytes) 2024-09-14T07:00:08.557 INFO:teuthology.orchestra.run.smithi007.stdout:2296 punch from 0x28fb83c to 0x29037e6, (0x7faa bytes) 2024-09-14T07:00:08.557 INFO:teuthology.orchestra.run.smithi007.stdout:2297 punch from 0x7b4582c to 0x7b493a9, (0x3b7d bytes) 2024-09-14T07:00:08.568 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:08.566+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:08.570 INFO:teuthology.orchestra.run.smithi007.stdout:2298 trunc from 0xdbc8534 to 0x2813648 2024-09-14T07:00:08.613 INFO:teuthology.orchestra.run.smithi007.stdout:2299 punch from 0x1f228b3 to 0x1f27250, (0x499d bytes) 2024-09-14T07:00:08.613 INFO:teuthology.orchestra.run.smithi007.stdout:2300 read 0x191861 thru 0x19e9aa (0xd14a bytes) 2024-09-14T07:00:08.614 INFO:teuthology.orchestra.run.smithi007.stdout:2301 punch from 0x1d81fa7 to 0x1d8e1e9, (0xc242 bytes) 2024-09-14T07:00:08.614 INFO:teuthology.orchestra.run.smithi007.stdout:2302 read 0x170b59c thru 0x171577f (0xa1e4 bytes) 2024-09-14T07:00:08.614 INFO:teuthology.orchestra.run.smithi007.stdout:2303 read 0x3838f0 thru 0x3930c8 (0xf7d9 bytes) 2024-09-14T07:00:08.617 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:08.617 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:08.618 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:08.618 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:08.625 INFO:teuthology.orchestra.run.smithi007.stdout:2304 writesame 0x6acc6e2 thru 0x6ad0d11 data_size 0x8(0x4630 bytes) 2024-09-14T07:00:08.626 INFO:teuthology.orchestra.run.smithi007.stdout:2305 punch from 0x2d869ff to 0x2d9594f, (0xef50 bytes) 2024-09-14T07:00:08.626 INFO:teuthology.orchestra.run.smithi007.stdout:2307 read 0x369c17 thru 0x36d967 (0x3d51 bytes) 2024-09-14T07:00:08.627 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:08.632 INFO:teuthology.orchestra.run.smithi007.stdout:2308 write 0x726abf3 thru 0x7279487 (0xe895 bytes) 2024-09-14T07:00:08.635 INFO:teuthology.orchestra.run.smithi007.stdout:2310 read 0x42b2a18 thru 0x42bd7de (0xadc7 bytes) 2024-09-14T07:00:08.635 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.643 INFO:teuthology.orchestra.run.smithi007.stdout:2312 writesame 0xa03f4b8 thru 0xa044473 data_size 0x6c(0x4fbc bytes) 2024-09-14T07:00:08.644 INFO:teuthology.orchestra.run.smithi007.stdout:2313 read 0x766355a thru 0x7665590 (0x2037 bytes) 2024-09-14T07:00:08.647 INFO:teuthology.orchestra.run.smithi007.stdout:2314 trunc from 0xa044474 to 0x715bdbd 2024-09-14T07:00:08.660 INFO:teuthology.orchestra.run.smithi007.stdout:2315 trunc from 0x715bdbd to 0xc0f63f1 2024-09-14T07:00:08.668 INFO:teuthology.orchestra.run.smithi007.stdout:2316 trunc from 0xc0f63f1 to 0xdd2bf9b 2024-09-14T07:00:08.669 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:08.668+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:08.673 INFO:teuthology.orchestra.run.smithi007.stdout:2318 read 0xc31f35d thru 0xc3215ed (0x2291 bytes) 2024-09-14T07:00:08.674 INFO:teuthology.orchestra.run.smithi007.stdout:2319 write 0x7a7b0b1 thru 0x7a7fc81 (0x4bd1 bytes) 2024-09-14T07:00:08.677 INFO:teuthology.orchestra.run.smithi007.stdout:2320 write 0xb43283c thru 0xb437762 (0x4f27 bytes) 2024-09-14T07:00:08.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:08.681 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.681 INFO:teuthology.orchestra.run.smithi007.stdout:2323 read 0xb7484a3 thru 0xb751f37 (0x9a95 bytes) 2024-09-14T07:00:08.681 INFO:teuthology.orchestra.run.smithi007.stdout:2325 read 0x10196dd thru 0x101d960 (0x4284 bytes) 2024-09-14T07:00:08.682 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.682 INFO:teuthology.orchestra.run.smithi007.stdout:2328 writesame 0x14cb632 thru 0x14d40f2 data_size 0x1(0x8ac1 bytes) 2024-09-14T07:00:08.697 INFO:teuthology.orchestra.run.smithi007.stdout:2329 read 0x911da thru 0x9b8e2 (0xa709 bytes) 2024-09-14T07:00:08.698 INFO:teuthology.orchestra.run.smithi007.stdout:2330 write 0x4bc98a4 thru 0x4bcb9d5 (0x2132 bytes) 2024-09-14T07:00:08.705 INFO:teuthology.orchestra.run.smithi007.stdout:2332 trunc from 0xdd2bf9b to 0xabdb6b6 2024-09-14T07:00:08.721 INFO:teuthology.orchestra.run.smithi007.stdout:2334 write 0xc0e30ae thru 0xc0eb978 (0x88cb bytes) 2024-09-14T07:00:08.723 INFO:teuthology.orchestra.run.smithi007.stdout:2335 punch from 0x205b160 to 0x205f480, (0x4320 bytes) 2024-09-14T07:00:08.723 INFO:teuthology.orchestra.run.smithi007.stdout:2336 write 0x30b092a thru 0x30bb7d8 (0xaeaf bytes) 2024-09-14T07:00:08.727 INFO:teuthology.orchestra.run.smithi007.stdout:2337 punch from 0x2545547 to 0x2547c76, (0x272f bytes) 2024-09-14T07:00:08.727 INFO:teuthology.orchestra.run.smithi007.stdout:2338 write 0xc45e66 thru 0xc4d169 (0x7304 bytes) 2024-09-14T07:00:08.729 INFO:teuthology.orchestra.run.smithi007.stdout:2339 punch from 0x25e02b3 to 0x25ed2b2, (0xcfff bytes) 2024-09-14T07:00:08.729 INFO:teuthology.orchestra.run.smithi007.stdout:2340 write 0x4916217 thru 0x49255d7 (0xf3c1 bytes) 2024-09-14T07:00:08.731 INFO:teuthology.orchestra.run.smithi007.stdout:2341 read 0x6311e16 thru 0x631391c (0x1b07 bytes) 2024-09-14T07:00:08.732 INFO:teuthology.orchestra.run.smithi007.stdout:2342 punch from 0x1ca57da to 0x1cac5b0, (0x6dd6 bytes) 2024-09-14T07:00:08.732 INFO:teuthology.orchestra.run.smithi007.stdout:2343 write 0x3ee8ffb thru 0x3eefd42 (0x6d48 bytes) 2024-09-14T07:00:08.733 INFO:teuthology.orchestra.run.smithi007.stdout:2350 read 0xcaa37d thru 0xcb8aaf (0xe733 bytes) 2024-09-14T07:00:08.734 INFO:teuthology.orchestra.run.smithi007.stdout:2351 read 0x9ec1d37 thru 0x9ecf96d (0xdc37 bytes) 2024-09-14T07:00:08.734 INFO:teuthology.orchestra.run.smithi007.stdout:2353 punch from 0x512fc93 to 0x513357c, (0x38e9 bytes) 2024-09-14T07:00:08.734 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.734 INFO:teuthology.orchestra.run.smithi007.stdout:2355 read 0x1c74587 thru 0x1c802a2 (0xbd1c bytes) 2024-09-14T07:00:08.735 INFO:teuthology.orchestra.run.smithi007.stdout:2356 punch from 0xbb74ea6 to 0xbb84a8c, (0xfbe6 bytes) 2024-09-14T07:00:08.735 INFO:teuthology.orchestra.run.smithi007.stdout:2357 write 0x6adbb8d thru 0x6ae1d3b (0x61af bytes) 2024-09-14T07:00:08.752 INFO:teuthology.orchestra.run.smithi007.stdout:2359 trunc from 0xc0eb979 to 0x10a868 2024-09-14T07:00:08.769 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:08.768+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:08.773 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:08.773 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:08.773 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:08.773 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:08.774 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:08.779 INFO:teuthology.orchestra.run.smithi007.stdout:2360 write 0x1871122 thru 0x188022f (0xf10e bytes) 2024-09-14T07:00:08.782 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:08.784 INFO:teuthology.orchestra.run.smithi007.stdout:2361 read 0x105e68d thru 0x106b2f6 (0xcc6a bytes) 2024-09-14T07:00:08.784 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.791 INFO:teuthology.orchestra.run.smithi007.stdout:2363 write 0x268a133 thru 0x2698d7b (0xec49 bytes) 2024-09-14T07:00:08.795 INFO:teuthology.orchestra.run.smithi007.stdout:2365 trunc from 0x2698d7c to 0x308602 2024-09-14T07:00:08.805 INFO:teuthology.orchestra.run.smithi007.stdout:2366 punch from 0x258fc4 to 0x25dd07, (0x4d43 bytes) 2024-09-14T07:00:08.805 INFO:teuthology.orchestra.run.smithi007.stdout:2367 read 0xa218f thru 0xa66ee (0x4560 bytes) 2024-09-14T07:00:08.817 INFO:teuthology.orchestra.run.smithi007.stdout:2368 write 0x3e3090f thru 0x3e3eb9b (0xe28d bytes) 2024-09-14T07:00:08.821 INFO:teuthology.orchestra.run.smithi007.stdout:2369 read 0x1e5d0cf thru 0x1e6127b (0x41ad bytes) 2024-09-14T07:00:08.838 INFO:teuthology.orchestra.run.smithi007.stdout:2370 write 0xd7148c8 thru 0xd715c71 (0x13aa bytes) 2024-09-14T07:00:08.841 INFO:teuthology.orchestra.run.smithi007.stdout:2371 writesame 0x74e567c thru 0x74e894a data_size 0x1(0x32cf bytes) 2024-09-14T07:00:08.845 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:08.849 INFO:teuthology.orchestra.run.smithi007.stdout:2373 read 0x99a57a0 thru 0x99a5d2c (0x58d bytes) 2024-09-14T07:00:08.849 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.858 INFO:teuthology.orchestra.run.smithi007.stdout:2378 write 0xedd6c15 thru 0xede203a (0xb426 bytes) 2024-09-14T07:00:08.859 INFO:teuthology.orchestra.run.smithi007.stdout:2379 writesame 0x8f5072d thru 0x8f51b19 data_size 0x1(0x13ed bytes) 2024-09-14T07:00:08.864 INFO:teuthology.orchestra.run.smithi007.stdout:2380 writesame 0x9f0af43 thru 0x9f0c837 data_size 0x1(0x18f5 bytes) 2024-09-14T07:00:08.867 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:08.867 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:08.870 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:08.868+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:08.883 INFO:teuthology.orchestra.run.smithi007.stdout:2382 trunc from 0xede203b to 0x3188ae0 2024-09-14T07:00:08.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:08.891 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:08.891 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8489 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8208384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.220000055, 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033127139, 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.892 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8489 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670276+0000", 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670277+0000", 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6109999999999999e-06 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670283+0000", 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7409999999999998e-06 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670287+0000", 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1679999999999997e-06 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703311+0000", 2024-09-14T07:00:08.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033023640999999999 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703352+0000", 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1168999999999998e-05 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703402+0000", 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0424999999999999e-05 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8490 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219982472000002, 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033303263, 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8490 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.898 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:08.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670295+0000", 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670300+0000", 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5369999999999997e-06 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670304+0000", 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8789999999999996e-06 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703433+0000", 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033128697999999998 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703510+0000", 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6662000000000006e-05 2024-09-14T07:00:08.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703596+0000", 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6614999999999997e-05 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8491 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8454144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219964842, 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033389611999999999, 2024-09-14T07:00:08.902 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8491 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4900000000000001e-07 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670317+0000", 2024-09-14T07:00:08.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.19e-06 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670323+0000", 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670331+0000", 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1280000000000008e-06 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703611+0000", 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033279557000000001 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703650+0000", 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9410999999999999e-05 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703700+0000", 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0117000000000001e-05 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.906 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8492 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8568832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219936791999999, 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033561793999999999, 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.907 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8492 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670341+0000", 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8700000000000001e-06 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670347+0000", 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000004e-06 2024-09-14T07:00:08.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670351+0000", 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4660000000000001e-06 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703731+0000", 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033379502999999998 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703812+0000", 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1192000000000005e-05 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703900+0000", 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8382999999999997e-05 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8493 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8683520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:08.911 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219918293999999, 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033663516999999997, 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8493 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:08.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670359+0000", 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.685e-06 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670365+0000", 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.57e-06 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670369+0000", 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4460000000000003e-06 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703928+0000", 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033558530000000003 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703970+0000", 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1909999999999997e-05 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704021+0000", 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0988e-05 2024-09-14T07:00:08.915 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8494 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8798208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219888697999998, 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033828579999999997, 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.916 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8494 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670389+0000", 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.64e-06 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670393+0000", 2024-09-14T07:00:08.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3470000000000001e-06 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670396+0000", 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6369999999999999e-06 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704051+0000", 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033655293000000003 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704128+0000", 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7453000000000005e-05 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704215+0000", 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6884000000000004e-05 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8495 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8912896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.920 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219875148, 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033930725000000002, 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8495 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670401+0000", 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.15e-07 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670402+0000", 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.850000000000001e-07 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670405+0000", 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.225e-06 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670407+0000", 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.34e-06 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704240+0000", 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033833019999999998 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704281+0000", 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1332999999999999e-05 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704331+0000", 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9506999999999999e-05 2024-09-14T07:00:08.924 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8496 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9027584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219863824000001, 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034114617, 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8496 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670413+0000", 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8999999999999995e-07 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670416+0000", 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2729999999999998e-06 2024-09-14T07:00:08.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670418+0000", 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3709999999999998e-06 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704362+0000", 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033943917999999997 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704439+0000", 2024-09-14T07:00:08.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6922999999999995e-05 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704526+0000", 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7035999999999998e-05 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8497 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9142272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219853936, 2024-09-14T07:00:08.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034220315000000001, 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8497 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:08.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670422+0000", 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670426+0000", 2024-09-14T07:00:08.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5029999999999999e-06 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670430+0000", 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-06 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670432+0000", 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704552+0000", 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034120277999999997 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704592+0000", 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9884e-05 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704642+0000", 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9623999999999998e-05 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.933 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8498 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9256960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219840173000001, 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034407434000000001, 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8498 2024-09-14T07:00:08.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670436+0000", 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670439+0000", 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670442+0000", 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3120000000000001e-06 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704673+0000", 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034231175000000003 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704749+0000", 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6069999999999995e-05 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704843+0000", 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3622000000000006e-05 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8499 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9371648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219830652999999, 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034523731000000002, 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.938 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8499 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2000000000000001e-07 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670446+0000", 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670449+0000", 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809999999999998e-06 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670455+0000", 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7930000000000003e-06 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704869+0000", 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034413931000000002 2024-09-14T07:00:08.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704910+0000", 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1032999999999998e-05 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704968+0000", 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8897999999999998e-05 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.942 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8500 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9486336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219817376000002, 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034708731, 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8500 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.943 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3300000000000001e-07 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670459+0000", 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670462+0000", 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1350000000000001e-06 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670470+0000", 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7400000000000004e-06 2024-09-14T07:00:08.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705002+0000", 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034531470000000002 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705079+0000", 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7472999999999995e-05 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705167+0000", 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7750000000000005e-05 2024-09-14T07:00:08.946 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8501 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9601024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219798947000001, 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034806278000000003, 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.947 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8501 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670477+0000", 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.44e-07 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670479+0000", 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7740000000000001e-06 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670485+0000", 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9150000000000001e-06 2024-09-14T07:00:08.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670489+0000", 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5469999999999998e-06 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705191+0000", 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034701896000000003 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705234+0000", 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2394000000000001e-05 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705283+0000", 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9208e-05 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8502 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9715712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219778607999999, 2024-09-14T07:00:08.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034979270999999999, 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8502 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670506+0000", 2024-09-14T07:00:08.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9740000000000008e-06 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670512+0000", 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670517+0000", 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7049999999999998e-06 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705313+0000", 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034795858999999998 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705391+0000", 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8166999999999999e-05 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705476+0000", 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5285999999999996e-05 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8503 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9830400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219752186000001, 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053094, 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8503 2024-09-14T07:00:08.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670524+0000", 2024-09-14T07:00:08.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:08.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670525+0000", 2024-09-14T07:00:08.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.581e-06 2024-09-14T07:00:08.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670531+0000", 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8969999999999996e-06 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670536+0000", 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4630000000000003e-06 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705488+0000", 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034952451000000002 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705526+0000", 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8396999999999998e-05 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705576+0000", 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9926000000000003e-05 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8504 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9945088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219733848000001, 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035228601999999998, 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8504 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.962 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:08.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670543+0000", 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5820000000000001e-06 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670549+0000", 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670554+0000", 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3699999999999997e-06 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705607+0000", 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053594 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705685+0000", 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8102999999999994e-05 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705770+0000", 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4678000000000005e-05 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8505 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10059776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219716061, 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035318548999999998, 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.966 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8505 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670560+0000", 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670561+0000", 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670568+0000", 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9789999999999996e-06 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670572+0000", 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2509999999999996e-06 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705784+0000", 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035211724 2024-09-14T07:00:08.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705827+0000", 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3276999999999999e-05 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705878+0000", 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0488000000000002e-05 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8506 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10174464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219695975, 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035495512999999999, 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8506 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670580+0000", 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:08.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670582+0000", 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670587+0000", 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.767e-06 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670592+0000", 2024-09-14T07:00:08.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2819999999999998e-06 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705909+0000", 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.03531753 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705987+0000", 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7787999999999995e-05 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706075+0000", 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7998999999999996e-05 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8507 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10289152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219678149, 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035582134000000001, 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.975 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8507 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:08.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670598+0000", 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9900000000000002e-07 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670599+0000", 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6029999999999999e-06 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670609+0000", 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.004e-05 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670614+0000", 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0749999999999997e-06 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706089+0000", 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035474358999999997 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706129+0000", 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0204999999999999e-05 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706179+0000", 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0553000000000002e-05 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8508 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10403840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:08.979 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 25.219654519999999, 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035743559000000001, 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8508 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:08.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2800000000000002e-07 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:08.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670623+0000", 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670629+0000", 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.02e-06 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670634+0000", 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6280000000000004e-06 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706200+0000", 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035566564000000002 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706276+0000", 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5801000000000002e-05 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706364+0000", 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8176000000000004e-05 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.983 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.984 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.984 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:08.984 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:08.984 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:08.985 INFO:teuthology.orchestra.run.smithi007.stdout:2383 trunc from 0x3188ae0 to 0xbdb0222 2024-09-14T07:00:08.985 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:08.986 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:08.983+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:08.987 INFO:teuthology.orchestra.run.smithi007.stdout:2384 write 0xe3ee3fe thru 0xe3fc135 (0xdd38 bytes) 2024-09-14T07:00:08.987 INFO:teuthology.orchestra.run.smithi007.stdout:2385 write 0x45674e7 thru 0x456fdf8 (0x8912 bytes) 2024-09-14T07:00:08.987 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.987 INFO:teuthology.orchestra.run.smithi007.stdout:2388 read 0x6b05ac thru 0x6b2a2f (0x2484 bytes) 2024-09-14T07:00:08.987 INFO:teuthology.orchestra.run.smithi007.stdout:2389 write 0x8816fcd thru 0x88229f7 (0xba2b bytes) 2024-09-14T07:00:08.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:08.988 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:08.988 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:08.989 INFO:teuthology.orchestra.run.smithi007.stdout:2391 read 0xc40451a thru 0xc407a72 (0x3559 bytes) 2024-09-14T07:00:08.989 INFO:teuthology.orchestra.run.smithi007.stdout:2392 read 0xb44b512 thru 0xb45aeb7 (0xf9a6 bytes) 2024-09-14T07:00:08.989 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.989 INFO:teuthology.orchestra.run.smithi007.stdout:2396 writesame 0x282f1a0 thru 0x28384e1 data_size 0x6(0x9342 bytes) 2024-09-14T07:00:08.989 INFO:teuthology.orchestra.run.smithi007.stdout:2397 write 0x2f167ca thru 0x2f1d32d (0x6b64 bytes) 2024-09-14T07:00:08.989 INFO:teuthology.orchestra.run.smithi007.stdout:2399 trunc from 0xe3fc136 to 0xb9ce412 2024-09-14T07:00:08.989 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:08.990 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:08.991 INFO:teuthology.orchestra.run.smithi007.stdout:2400 writesame 0xf878fa thru 0xf9573f data_size 0x17(0xde46 bytes) 2024-09-14T07:00:08.991 INFO:teuthology.orchestra.run.smithi007.stdout:2401 writesame 0x640d117 thru 0x6415ffc data_size 0x5b(0x8ee6 bytes) 2024-09-14T07:00:08.991 INFO:teuthology.orchestra.run.smithi007.stdout:2402 writesame 0x5ed97d5 thru 0x5edf4e4 data_size 0x8(0x5d10 bytes) 2024-09-14T07:00:08.991 INFO:teuthology.orchestra.run.smithi007.stdout:2404 read 0x8feb10a thru 0x8feef1b (0x3e12 bytes) 2024-09-14T07:00:08.991 INFO:teuthology.orchestra.run.smithi007.stdout:2405 read 0xa2e2945 thru 0xa2e92a2 (0x695e bytes) 2024-09-14T07:00:08.991 INFO:teuthology.orchestra.run.smithi007.stdout:2406 read 0xaef1c9d thru 0xaef2daa (0x110e bytes) 2024-09-14T07:00:08.991 INFO:teuthology.orchestra.run.smithi007.stdout:2407 writesame 0x44bb494 thru 0x44bf081 data_size 0x3(0x3bee bytes) 2024-09-14T07:00:08.991 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:08.992 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:08.992 INFO:teuthology.orchestra.run.smithi007.stdout:2408 trunc from 0xb9ce412 to 0x66af2dc 2024-09-14T07:00:08.992 INFO:teuthology.orchestra.run.smithi007.stdout:2409 read 0x2585527 thru 0x25929fd (0xd4d7 bytes) 2024-09-14T07:00:08.992 INFO:teuthology.orchestra.run.smithi007.stdout:2410 read 0x52dd249 thru 0x52e3c36 (0x69ee bytes) 2024-09-14T07:00:08.993 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:08.993 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:08.994 INFO:teuthology.orchestra.run.smithi007.stdout:2412 write 0xb11b54c thru 0xb127bb5 (0xc66a bytes) 2024-09-14T07:00:08.994 INFO:teuthology.orchestra.run.smithi007.stdout:2413 read 0xa2d777b thru 0xa2e0833 (0x90b9 bytes) 2024-09-14T07:00:08.994 INFO:teuthology.orchestra.run.smithi007.stdout:2414 read 0x8bdbb13 thru 0x8be4722 (0x8c10 bytes) 2024-09-14T07:00:08.994 INFO:teuthology.orchestra.run.smithi007.stdout:2415 read 0xa457d4d thru 0xa45cf6b (0x521f bytes) 2024-09-14T07:00:08.994 INFO:teuthology.orchestra.run.smithi007.stdout:2417 read 0x6d42359 thru 0x6d47400 (0x50a8 bytes) 2024-09-14T07:00:08.994 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.994 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:08.994 INFO:teuthology.orchestra.run.smithi007.stdout:2421 read 0x448ffdb thru 0x4497d8b (0x7db1 bytes) 2024-09-14T07:00:08.994 INFO:teuthology.orchestra.run.smithi007.stdout:2422 writesame 0x75f81e3 thru 0x75f8471 data_size 0x1(0x28f bytes) 2024-09-14T07:00:08.995 INFO:teuthology.orchestra.run.smithi007.stdout:2424 write 0x2616a3c thru 0x261e46d (0x7a32 bytes) 2024-09-14T07:00:08.997 INFO:teuthology.orchestra.run.smithi007.stdout:2425 trunc from 0xb127bb6 to 0xa2ab8a9 2024-09-14T07:00:09.003 INFO:teuthology.orchestra.run.smithi007.stdout:2426 read 0x5c8ae3f thru 0x5c8c364 (0x1526 bytes) 2024-09-14T07:00:09.004 INFO:teuthology.orchestra.run.smithi007.stdout:2427 writesame 0x1d3c81a thru 0x1d3cb6c data_size 0x1(0x353 bytes) 2024-09-14T07:00:09.005 INFO:teuthology.orchestra.run.smithi007.stdout:2428 punch from 0x54eaace to 0x54fa9bc, (0xfeee bytes) 2024-09-14T07:00:09.005 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.006 INFO:teuthology.orchestra.run.smithi007.stdout:2434 read 0x93ef434 thru 0x93f7673 (0x8240 bytes) 2024-09-14T07:00:09.012 INFO:teuthology.orchestra.run.smithi007.stdout:2435 trunc from 0xa2ab8a9 to 0x48d61d5 2024-09-14T07:00:09.023 INFO:teuthology.orchestra.run.smithi007.stdout:2436 punch from 0x33f5efb to 0x33f917d, (0x3282 bytes) 2024-09-14T07:00:09.024 INFO:teuthology.orchestra.run.smithi007.stdout:2437 writesame 0x34b5e4 thru 0x353586 data_size 0x5(0x7fa3 bytes) 2024-09-14T07:00:09.042 INFO:teuthology.orchestra.run.smithi007.stdout:2439 writesame 0xca52458 thru 0xca570a2 data_size 0x1(0x4c4b bytes) 2024-09-14T07:00:09.042 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.042 INFO:teuthology.orchestra.run.smithi007.stdout:2441 read 0x71aecc7 thru 0x71b2505 (0x383f bytes) 2024-09-14T07:00:09.042 INFO:teuthology.orchestra.run.smithi007.stdout:2442 writesame 0x44bbdac thru 0x44c80e1 data_size 0x3(0xc336 bytes) 2024-09-14T07:00:09.050 INFO:teuthology.orchestra.run.smithi007.stdout:2443 read 0x77cb28e thru 0x77d68dc (0xb64f bytes) 2024-09-14T07:00:09.050 INFO:teuthology.orchestra.run.smithi007.stdout:2444 read 0x4b59cd5 thru 0x4b60a71 (0x6d9d bytes) 2024-09-14T07:00:09.050 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.050 INFO:teuthology.orchestra.run.smithi007.stdout:2447 writesame 0x3ce072 thru 0x3d1d31 data_size 0x9(0x3cc0 bytes) 2024-09-14T07:00:09.052 INFO:teuthology.orchestra.run.smithi007.stdout:2448 read 0x148c97e thru 0x1495147 (0x87ca bytes) 2024-09-14T07:00:09.052 INFO:teuthology.orchestra.run.smithi007.stdout:2450 read 0xc4af9e thru 0xc52755 (0x77b8 bytes) 2024-09-14T07:00:09.052 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.052 INFO:teuthology.orchestra.run.smithi007.stdout:2455 read 0x3b3cb86 thru 0x3b4c945 (0xfdc0 bytes) 2024-09-14T07:00:09.053 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.053 INFO:teuthology.orchestra.run.smithi007.stdout:2458 write 0xbf48aa thru 0xbfa398 (0x5aef bytes) 2024-09-14T07:00:09.055 INFO:teuthology.orchestra.run.smithi007.stdout:2459 write 0x8a8ad85 thru 0x8a8cc76 (0x1ef2 bytes) 2024-09-14T07:00:09.058 INFO:teuthology.orchestra.run.smithi007.stdout:2460 read 0x7defda9 thru 0x7df09f1 (0xc49 bytes) 2024-09-14T07:00:09.058 INFO:teuthology.orchestra.run.smithi007.stdout:2462 write 0x1578abb thru 0x15807b9 (0x7cff bytes) 2024-09-14T07:00:09.059 INFO:teuthology.orchestra.run.smithi007.stdout:2463 read 0x41fcc85 thru 0x41fe734 (0x1ab0 bytes) 2024-09-14T07:00:09.059 INFO:teuthology.orchestra.run.smithi007.stdout:2465 read 0xa93b675 thru 0xa943d26 (0x86b2 bytes) 2024-09-14T07:00:09.060 INFO:teuthology.orchestra.run.smithi007.stdout:2466 trunc from 0xca570a3 to 0xc83d0f8 2024-09-14T07:00:09.076 INFO:teuthology.orchestra.run.smithi007.stdout:2467 trunc from 0xc83d0f8 to 0x36eb83f 2024-09-14T07:00:09.085 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:09.084+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:09.087 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.087 INFO:teuthology.orchestra.run.smithi007.stdout:2469 punch from 0x22a7a09 to 0x22b2fbf, (0xb5b6 bytes) 2024-09-14T07:00:09.087 INFO:teuthology.orchestra.run.smithi007.stdout:2470 read 0x10d482c thru 0x10dafb6 (0x678b bytes) 2024-09-14T07:00:09.100 INFO:teuthology.orchestra.run.smithi007.stdout:2472 write 0xa9b69b1 thru 0xa9ba473 (0x3ac3 bytes) 2024-09-14T07:00:09.103 INFO:teuthology.orchestra.run.smithi007.stdout:2473 read 0x75cdf4f thru 0x75d3565 (0x5617 bytes) 2024-09-14T07:00:09.113 INFO:teuthology.orchestra.run.smithi007.stdout:2474 trunc from 0xa9ba474 to 0x208a14c 2024-09-14T07:00:09.121 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:09.121 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:09.127 INFO:teuthology.orchestra.run.smithi007.stdout:2475 trunc from 0x208a14c to 0x253d6d 2024-09-14T07:00:09.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:09.148 INFO:teuthology.orchestra.run.smithi007.stdout:2476 write 0x8b88b30 thru 0x8b8a319 (0x17ea bytes) 2024-09-14T07:00:09.161 INFO:teuthology.orchestra.run.smithi007.stdout:2477 write 0xc26da99 thru 0xc27c5f1 (0xeb59 bytes) 2024-09-14T07:00:09.163 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.169 INFO:teuthology.orchestra.run.smithi007.stdout:2479 writesame 0xcaef15f thru 0xcaf4cb5 data_size 0x1(0x5b57 bytes) 2024-09-14T07:00:09.170 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.181 INFO:teuthology.orchestra.run.smithi007.stdout:2484 trunc from 0xcaf4cb6 to 0x24e2b5b 2024-09-14T07:00:09.186 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:09.185+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:09.203 INFO:teuthology.orchestra.run.smithi007.stdout:2485 writesame 0x58bfff8 thru 0x58c8823 data_size 0x1e(0x882c bytes) 2024-09-14T07:00:09.203 INFO:teuthology.orchestra.run.smithi007.stdout:2486 read 0x15b4664 thru 0x15bc701 (0x809e bytes) 2024-09-14T07:00:09.221 INFO:teuthology.orchestra.run.smithi007.stdout:2487 writesame 0xd9ea058 thru 0xd9f5dcb data_size 0x7d(0xbd74 bytes) 2024-09-14T07:00:09.221 INFO:teuthology.orchestra.run.smithi007.stdout:2488 writesame 0xcac8783 thru 0xcacd1c2 data_size 0x48(0x4a40 bytes) 2024-09-14T07:00:09.223 INFO:teuthology.orchestra.run.smithi007.stdout:2490 write 0x7c97d83 thru 0x7c9a967 (0x2be5 bytes) 2024-09-14T07:00:09.226 INFO:teuthology.orchestra.run.smithi007.stdout:2491 read 0xa93d8d8 thru 0xa944631 (0x6d5a bytes) 2024-09-14T07:00:09.226 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.226 INFO:teuthology.orchestra.run.smithi007.stdout:2493 read 0x211b860 thru 0x211ff11 (0x46b2 bytes) 2024-09-14T07:00:09.232 INFO:teuthology.orchestra.run.smithi007.stdout:2495 write 0xdd063d7 thru 0xdd06a7f (0x6a9 bytes) 2024-09-14T07:00:09.235 INFO:teuthology.orchestra.run.smithi007.stdout:2496 punch from 0x2ea032a to 0x2eafb3e, (0xf814 bytes) 2024-09-14T07:00:09.237 INFO:teuthology.orchestra.run.smithi007.stdout:2497 trunc from 0xdd06a80 to 0xbe08b3d 2024-09-14T07:00:09.253 INFO:teuthology.orchestra.run.smithi007.stdout:2499 write 0xc73c126 thru 0xc740dc5 (0x4ca0 bytes) 2024-09-14T07:00:09.255 INFO:teuthology.orchestra.run.smithi007.stdout:2500 write 0x8008187 thru 0x8013498 (0xb312 bytes) 2024-09-14T07:00:09.258 INFO:teuthology.orchestra.run.smithi007.stdout:2501 write 0x72c157b thru 0x72c2817 (0x129d bytes) 2024-09-14T07:00:09.261 INFO:teuthology.orchestra.run.smithi007.stdout:2502 punch from 0xa0f603e to 0xa1052ab, (0xf26d bytes) 2024-09-14T07:00:09.261 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.265 INFO:teuthology.orchestra.run.smithi007.stdout:2508 trunc from 0xc740dc6 to 0x8dced75 2024-09-14T07:00:09.276 INFO:teuthology.orchestra.run.smithi007.stdout:2509 writesame 0x1bfb3f6 thru 0x1bff58d data_size 0x4(0x4198 bytes) 2024-09-14T07:00:09.281 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:09.281 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:09.287 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:09.286+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:09.289 INFO:teuthology.orchestra.run.smithi007.stdout:2510 trunc from 0x8dced75 to 0x7d9b28 2024-09-14T07:00:09.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:09.299 INFO:teuthology.orchestra.run.smithi007.stdout:2511 read 0x2f28af thru 0x300f6a (0xe6bc bytes) 2024-09-14T07:00:09.322 INFO:teuthology.orchestra.run.smithi007.stdout:2512 write 0xe8d74ff thru 0xe8dd259 (0x5d5b bytes) 2024-09-14T07:00:09.325 INFO:teuthology.orchestra.run.smithi007.stdout:2513 punch from 0x3edea62 to 0x3ee8f2e, (0xa4cc bytes) 2024-09-14T07:00:09.325 INFO:teuthology.orchestra.run.smithi007.stdout:2514 write 0x676463b thru 0x676c4c0 (0x7e86 bytes) 2024-09-14T07:00:09.328 INFO:teuthology.orchestra.run.smithi007.stdout:2516 write 0x9d4f0dc thru 0x9d5167c (0x25a1 bytes) 2024-09-14T07:00:09.332 INFO:teuthology.orchestra.run.smithi007.stdout:2517 write 0xd0ae454 thru 0xd0bb926 (0xd4d3 bytes) 2024-09-14T07:00:09.335 INFO:teuthology.orchestra.run.smithi007.stdout:2518 punch from 0x295cc3a to 0x2966f47, (0xa30d bytes) 2024-09-14T07:00:09.335 INFO:teuthology.orchestra.run.smithi007.stdout:2519 write 0x706af38 thru 0x707203e (0x7107 bytes) 2024-09-14T07:00:09.338 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.354 INFO:teuthology.orchestra.run.smithi007.stdout:2523 trunc from 0xe8dd25a to 0x9aff0a 2024-09-14T07:00:09.365 INFO:teuthology.orchestra.run.smithi007.stdout:2524 write 0x48ac44 thru 0x48e99a (0x3d57 bytes) 2024-09-14T07:00:09.367 INFO:teuthology.orchestra.run.smithi007.stdout:2525 read 0x158917 thru 0x16111c (0x8806 bytes) 2024-09-14T07:00:09.367 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.367 INFO:teuthology.orchestra.run.smithi007.stdout:2528 punch from 0xdd20 to 0x1bc96, (0xdf76 bytes) 2024-09-14T07:00:09.387 INFO:teuthology.orchestra.run.smithi007.stdout:2529 write 0xce82618 thru 0xce8be32 (0x981b bytes) 2024-09-14T07:00:09.388 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:09.387+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:09.397 INFO:teuthology.orchestra.run.smithi007.stdout:2530 write 0xe7b5f6b thru 0xe7c3d09 (0xdd9f bytes) 2024-09-14T07:00:09.398 INFO:teuthology.orchestra.run.smithi007.stdout:2531 read 0xa3fc4e0 thru 0xa405e5d (0x997e bytes) 2024-09-14T07:00:09.398 INFO:teuthology.orchestra.run.smithi007.stdout:2532 write 0x68823e6 thru 0x6887d7b (0x5996 bytes) 2024-09-14T07:00:09.402 INFO:teuthology.orchestra.run.smithi007.stdout:2535 writesame 0x4ecf58a thru 0x4edb37f data_size 0xf(0xbdf6 bytes) 2024-09-14T07:00:09.406 INFO:teuthology.orchestra.run.smithi007.stdout:2536 read 0x22039bf thru 0x2211a07 (0xe049 bytes) 2024-09-14T07:00:09.407 INFO:teuthology.orchestra.run.smithi007.stdout:2538 writesame 0xde88eca thru 0xde8f3a2 data_size 0x1(0x64d9 bytes) 2024-09-14T07:00:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.b pg_upmap_items mapping to [7->5,6->4] 2024-09-14T07:00:09.425 INFO:teuthology.orchestra.run.smithi007.stdout:2539 trunc from 0xe7c3d0a to 0x9a0c83b 2024-09-14T07:00:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T07:00:09.426 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:09.427 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:09.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:09.452 INFO:teuthology.orchestra.run.smithi007.stdout:2540 write 0xbf7df34 thru 0xbf8af76 (0xd043 bytes) 2024-09-14T07:00:09.457 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.465 INFO:teuthology.orchestra.run.smithi007.stdout:2543 write 0xe150912 thru 0xe153f55 (0x3644 bytes) 2024-09-14T07:00:09.471 INFO:teuthology.orchestra.run.smithi007.stdout:2544 trunc from 0xe153f56 to 0xb5ae7bc 2024-09-14T07:00:09.483 INFO:teuthology.orchestra.run.smithi007.stdout:2545 writesame 0x59b4f2f thru 0x59c0eae data_size 0x40(0xbf80 bytes) 2024-09-14T07:00:09.485 INFO:teuthology.orchestra.run.smithi007.stdout:2546 punch from 0x85ff101 to 0x860a354, (0xb253 bytes) 2024-09-14T07:00:09.485 INFO:teuthology.orchestra.run.smithi007.stdout:2547 punch from 0x24ff197 to 0x250ded4, (0xed3d bytes) 2024-09-14T07:00:09.485 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.485 INFO:teuthology.orchestra.run.smithi007.stdout:2550 punch from 0x8b6e282 to 0x8b72b4a, (0x48c8 bytes) 2024-09-14T07:00:09.485 INFO:teuthology.orchestra.run.smithi007.stdout:2551 read 0x633055 thru 0x63d3a6 (0xa352 bytes) 2024-09-14T07:00:09.486 INFO:teuthology.orchestra.run.smithi007.stdout:2552 punch from 0x1c45b51 to 0x1c4a4bf, (0x496e bytes) 2024-09-14T07:00:09.488 INFO:teuthology.orchestra.run.smithi007.stdout:2553 trunc from 0xb5ae7bc to 0xe1df215 2024-09-14T07:00:09.489 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:09.488+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:09.494 INFO:teuthology.orchestra.run.smithi007.stdout:2556 read 0xb1fe037 thru 0xb20a000 (0xbfca bytes) 2024-09-14T07:00:09.504 INFO:teuthology.orchestra.run.smithi007.stdout:2557 trunc from 0xe1df215 to 0x508afa6 2024-09-14T07:00:09.517 INFO:teuthology.orchestra.run.smithi007.stdout:2558 read 0x4ac477b thru 0x4ac7416 (0x2c9c bytes) 2024-09-14T07:00:09.530 INFO:teuthology.orchestra.run.smithi007.stdout:2561 write 0xc2a7673 thru 0xc2aa725 (0x30b3 bytes) 2024-09-14T07:00:09.533 INFO:teuthology.orchestra.run.smithi007.stdout:2562 writesame 0x7f8c70e thru 0x7f96405 data_size 0x4(0x9cf8 bytes) 2024-09-14T07:00:09.540 INFO:teuthology.orchestra.run.smithi007.stdout:2563 punch from 0x255828f to 0x255e50c, (0x627d bytes) 2024-09-14T07:00:09.540 INFO:teuthology.orchestra.run.smithi007.stdout:2564 punch from 0xb8abf2 to 0xb98ff6, (0xe404 bytes) 2024-09-14T07:00:09.547 INFO:teuthology.orchestra.run.smithi007.stdout:2565 trunc from 0xc2aa726 to 0x68b32b1 2024-09-14T07:00:09.556 INFO:teuthology.orchestra.run.smithi007.stdout:2566 write 0x31c9c45 thru 0x31d5494 (0xb850 bytes) 2024-09-14T07:00:09.565 INFO:teuthology.orchestra.run.smithi007.stdout:2567 write 0x6f76774 thru 0x6f84936 (0xe1c3 bytes) 2024-09-14T07:00:09.568 INFO:teuthology.orchestra.run.smithi007.stdout:2569 writesame 0x65cc10c thru 0x65cd852 data_size 0x1(0x1747 bytes) 2024-09-14T07:00:09.571 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:09.571 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:09.572 INFO:teuthology.orchestra.run.smithi007.stdout:2570 write 0x18951de thru 0x1898b47 (0x396a bytes) 2024-09-14T07:00:09.573 INFO:teuthology.orchestra.run.smithi007.stdout:2571 punch from 0xf3e938 to 0xf48d6e, (0xa436 bytes) 2024-09-14T07:00:09.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:09.585 INFO:teuthology.orchestra.run.smithi007.stdout:2572 write 0xbc72e53 thru 0xbc7ffc7 (0xd175 bytes) 2024-09-14T07:00:09.587 INFO:teuthology.orchestra.run.smithi007.stdout:2573 read 0x5cb8dde thru 0x5cbc23b (0x345e bytes) 2024-09-14T07:00:09.588 INFO:teuthology.orchestra.run.smithi007.stdout:2576 trunc from 0xbc7ffc8 to 0xc57aec0 2024-09-14T07:00:09.590 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:09.589+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:09.600 INFO:teuthology.orchestra.run.smithi007.stdout:2577 writesame 0xd02cecf thru 0xd032b00 data_size 0x1(0x5c32 bytes) 2024-09-14T07:00:09.600 INFO:teuthology.orchestra.run.smithi007.stdout:2579 read 0x3c11ed2 thru 0x3c1923e (0x736d bytes) 2024-09-14T07:00:09.600 INFO:teuthology.orchestra.run.smithi007.stdout:2580 punch from 0x23e2a2 to 0x24dff4, (0xfd52 bytes) 2024-09-14T07:00:09.600 INFO:teuthology.orchestra.run.smithi007.stdout:2581 punch from 0xcd447b to 0xce24dd, (0xe062 bytes) 2024-09-14T07:00:09.600 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.600 INFO:teuthology.orchestra.run.smithi007.stdout:2583 write 0x4853d5c thru 0x48560f6 (0x239b bytes) 2024-09-14T07:00:09.614 INFO:teuthology.orchestra.run.smithi007.stdout:2585 trunc from 0xd032b01 to 0x1b09135 2024-09-14T07:00:09.626 INFO:teuthology.orchestra.run.smithi007.stdout:2586 punch from 0x1108b5e to 0x1114381, (0xb823 bytes) 2024-09-14T07:00:09.626 INFO:teuthology.orchestra.run.smithi007.stdout:2588 read 0xcd7490 thru 0xcda244 (0x2db5 bytes) 2024-09-14T07:00:09.626 INFO:teuthology.orchestra.run.smithi007.stdout:2589 read 0x1648d3e thru 0x164eb8b (0x5e4e bytes) 2024-09-14T07:00:09.627 INFO:teuthology.orchestra.run.smithi007.stdout:2590 read 0xf615bc thru 0xf622b1 (0xcf6 bytes) 2024-09-14T07:00:09.627 INFO:teuthology.orchestra.run.smithi007.stdout:2591 read 0x9d8d2 thru 0xa8046 (0xa775 bytes) 2024-09-14T07:00:09.628 INFO:teuthology.orchestra.run.smithi007.stdout:2592 read 0xc579e5 thru 0xc64f39 (0xd555 bytes) 2024-09-14T07:00:09.628 INFO:teuthology.orchestra.run.smithi007.stdout:2593 punch from 0x14ac69b to 0x14b033f, (0x3ca4 bytes) 2024-09-14T07:00:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:2594 writesame 0x80b5f1b thru 0x80b9001 data_size 0xd(0x30e7 bytes) 2024-09-14T07:00:09.641 INFO:teuthology.orchestra.run.smithi007.stdout:2595 write 0x678e5d6 thru 0x678fd5d (0x1788 bytes) 2024-09-14T07:00:09.655 INFO:teuthology.orchestra.run.smithi007.stdout:2596 write 0xcfa7b24 thru 0xcfb26ad (0xab8a bytes) 2024-09-14T07:00:09.658 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.658 INFO:teuthology.orchestra.run.smithi007.stdout:2598 write 0x92614fd thru 0x926dc91 (0xc795 bytes) 2024-09-14T07:00:09.661 INFO:teuthology.orchestra.run.smithi007.stdout:2599 punch from 0x7e986d5 to 0x7ea7185, (0xeab0 bytes) 2024-09-14T07:00:09.661 INFO:teuthology.orchestra.run.smithi007.stdout:2600 read 0x18bc53b thru 0x18c616e (0x9c34 bytes) 2024-09-14T07:00:09.662 INFO:teuthology.orchestra.run.smithi007.stdout:2601 write 0x6684940 thru 0x66925d9 (0xdc9a bytes) 2024-09-14T07:00:09.671 INFO:teuthology.orchestra.run.smithi007.stdout:2602 write 0xd8071de thru 0xd80f996 (0x87b9 bytes) 2024-09-14T07:00:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:09.675 INFO:teuthology.orchestra.run.smithi007.stdout:2605 write 0x9d7b5e4 thru 0x9d7c2fe (0xd1b bytes) 2024-09-14T07:00:09.678 INFO:teuthology.orchestra.run.smithi007.stdout:2606 read 0x8ae1f3c thru 0x8af1877 (0xf93c bytes) 2024-09-14T07:00:09.679 INFO:teuthology.orchestra.run.smithi007.stdout:2608 punch from 0x856a265 to 0x8572a03, (0x879e bytes) 2024-09-14T07:00:09.679 INFO:teuthology.orchestra.run.smithi007.stdout:2609 read 0x54c645a thru 0x54cedf6 (0x899d bytes) 2024-09-14T07:00:09.691 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:09.690+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:09.693 INFO:teuthology.orchestra.run.smithi007.stdout:2610 trunc from 0xd80f997 to 0x2bad6c8 2024-09-14T07:00:09.716 INFO:teuthology.orchestra.run.smithi007.stdout:2611 writesame 0xa5160c5 thru 0xa518231 data_size 0x1(0x216d bytes) 2024-09-14T07:00:09.718 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:09.718 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:09.721 INFO:teuthology.orchestra.run.smithi007.stdout:2613 trunc from 0xa518232 to 0xe8e136f 2024-09-14T07:00:09.727 INFO:teuthology.orchestra.run.smithi007.stdout:2614 writesame 0x43cc9af thru 0x43cd406 data_size 0x4(0xa58 bytes) 2024-09-14T07:00:09.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:2615 read 0xc5a7180 thru 0xc5b0a6e (0x98ef bytes) 2024-09-14T07:00:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:2616 clone 18 order 18 su 1024 sc 14 2024-09-14T07:00:09.792 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:09.790+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:09.878 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:09.878 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:09.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:09.931 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:09.930+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:09.994 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:09.992+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:10.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:10.095 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:10.093+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:10.150 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:10.150 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:10.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:10.196 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:10.194+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:10.266 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:10.266 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:10.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:10.297 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:10.295+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:10.381 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:10.382 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:10.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:10.398 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:10.397+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:10.498 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:10.498+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:10.505 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:10.505 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:10.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:10.599 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:10.599+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:10.673 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:10.673 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:10.683 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:10.699 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-14T07:00:10.705 INFO:teuthology.orchestra.run.smithi007.stdout:2618 punch from 0xe16e28 to 0xe215d7, (0xa7af bytes) 2024-09-14T07:00:10.710 INFO:teuthology.orchestra.run.smithi007.stdout:2619 read 0xc95f57 thru 0xca0fa2 (0xb04c bytes) 2024-09-14T07:00:10.717 INFO:teuthology.orchestra.run.smithi007.stdout:2620 punch from 0xa2e808a to 0xa2ebf4c, (0x3ec2 bytes) 2024-09-14T07:00:10.717 INFO:teuthology.orchestra.run.smithi007.stdout:2621 write 0x29eadc3 thru 0x29f3f32 (0x9170 bytes) 2024-09-14T07:00:10.727 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:10.726+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:10.801 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:10.799+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:10.833 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:10.833 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:10.838 INFO:teuthology.orchestra.run.smithi007.stdout:2622 write 0x1f99976 thru 0x1fa6583 (0xcc0e bytes) 2024-09-14T07:00:10.844 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:10.928 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:10.926+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:10.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:11.025 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:11.025 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:11.034 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:11.041 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:11.041+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:11.120 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:11.118+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:11.123 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:11.123 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:11.140 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:11.141 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:11.157 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:11.173 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:11.173 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:11.178 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:11.178 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:11.186 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:11.191 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:11.208 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:11.225 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:11.225 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:11.227 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:11.226+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:11.242 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:11.307 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:11.305+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:11.330 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:11.330 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:11.330 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:11.330 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:11.339 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:11.423 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:11.421+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:11.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:00:11.484 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:11.484 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:11.484 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:11.484 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:11.484 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:11.493 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:11.506 INFO:teuthology.orchestra.run.smithi007.stdout:2623 read 0x75f54e8 thru 0x75fdee6 (0x89ff bytes) 2024-09-14T07:00:11.506 INFO:teuthology.orchestra.run.smithi007.stdout:2625 write 0xd0a140c thru 0xd0a5127 (0x3d1c bytes) 2024-09-14T07:00:11.526 INFO:teuthology.orchestra.run.smithi007.stdout:2626 write 0x8fc5771 thru 0x8fd4784 (0xf014 bytes) 2024-09-14T07:00:11.534 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:11.532+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:11.544 INFO:teuthology.orchestra.run.smithi007.stdout:2628 read 0xafa8dbd thru 0xafabbb5 (0x2df9 bytes) 2024-09-14T07:00:11.560 INFO:teuthology.orchestra.run.smithi007.stdout:2629 trunc from 0xe8e136f to 0x7583b6 2024-09-14T07:00:11.609 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:11.607+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:11.664 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:11.664 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:11.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:11.664 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:11.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.664 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:11.664 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.497477329999999, 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:11.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:00:11.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:00:11.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8461342250000001, 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:00:11.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:00:11.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8461187570000002, 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.673 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:00:11.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8461074559999999, 2024-09-14T07:00:11.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:00:11.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8393639830000001, 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.682 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.839351379, 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.686 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:11.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.839331863, 2024-09-14T07:00:11.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:00:11.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:00:11.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8393198220000002, 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:11.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:00:11.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.83929952, 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.699 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.700 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:00:11.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.700 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.704 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8392872040000001, 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8392757639999999, 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:00:11.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:00:11.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:00:11.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8392652649999999, 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:00:11.713 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:00:11.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:00:11.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8392550720000003, 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.721 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.839239525, 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:00:11.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:00:11.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:00:11.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:00:11.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8392099169999998, 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8391882669999999, 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.730 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.8391660280000002, 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.734 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105530 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 661504~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829342+0000", 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.833579769, 2024-09-14T07:00:11.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044060136, 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105530 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829342+0000", 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829342+0000", 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829343+0000", 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829344+0000", 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0920000000000001e-06 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829347+0000", 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.163e-06 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829350+0000", 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6189999999999998e-06 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873308+0000", 2024-09-14T07:00:11.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043958460999999997 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873349+0000", 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1414e-05 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873403+0000", 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3180999999999998e-05 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.833569663, 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.746 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 3.83355961, 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:11.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:00:11.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:00:11.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:00:11.750 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.751 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:11.751 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:11.751 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:11.752 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:11.753 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:11.751+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:11.795 INFO:teuthology.orchestra.run.smithi007.stdout:2631 read 0x2c7367 thru 0x2ce3b2 (0x704c bytes) 2024-09-14T07:00:11.801 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:11.811 INFO:teuthology.orchestra.run.smithi007.stdout:2634 writesame 0x4a7b28b thru 0x4a7e4e5 data_size 0x1(0x325b bytes) 2024-09-14T07:00:11.811 INFO:teuthology.orchestra.run.smithi007.stdout:2635 read 0xa91365 thru 0xa96cd6 (0x5972 bytes) 2024-09-14T07:00:11.811 INFO:teuthology.orchestra.run.smithi007.stdout:2636 writesame 0x2790ab9 thru 0x27917cc data_size 0x1b(0xd14 bytes) 2024-09-14T07:00:11.830 INFO:teuthology.orchestra.run.smithi007.stdout:2638 write 0xa99e829 thru 0xa9ad921 (0xf0f9 bytes) 2024-09-14T07:00:11.846 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:11.847 INFO:teuthology.orchestra.run.smithi007.stdout:2640 read 0x96bf91e thru 0x96c2270 (0x2953 bytes) 2024-09-14T07:00:11.847 INFO:teuthology.orchestra.run.smithi007.stdout:2641 write 0x995cde8 thru 0x9962263 (0x547c bytes) 2024-09-14T07:00:11.853 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:11.851+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:11.863 INFO:teuthology.orchestra.run.smithi007.stdout:2642 writesame 0x33922cd thru 0x33a081e data_size 0x2(0xe552 bytes) 2024-09-14T07:00:11.883 INFO:teuthology.orchestra.run.smithi007.stdout:2643 read 0x473be75 thru 0x473db6c (0x1cf8 bytes) 2024-09-14T07:00:11.883 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:11.883 INFO:teuthology.orchestra.run.smithi007.stdout:2645 read 0x4bce2c7 thru 0x4bd8094 (0x9dce bytes) 2024-09-14T07:00:11.883 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:11.890 INFO:teuthology.orchestra.run.smithi007.stdout:2648 writesame 0xcd9b501 thru 0xcd9e574 data_size 0xe(0x3074 bytes) 2024-09-14T07:00:11.891 INFO:teuthology.orchestra.run.smithi007.stdout:2649 read 0x20a226c thru 0x20adfeb (0xbd80 bytes) 2024-09-14T07:00:11.891 INFO:teuthology.orchestra.run.smithi007.stdout:2650 punch from 0x7e43e01 to 0x7e4b746, (0x7945 bytes) 2024-09-14T07:00:11.891 INFO:teuthology.orchestra.run.smithi007.stdout:2651 write 0x8e22bb4 thru 0x8e2b1ea (0x8637 bytes) 2024-09-14T07:00:11.908 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:11.908 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:11.909 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:11.909 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:11.911 INFO:teuthology.orchestra.run.smithi007.stdout:2652 read 0x225f22f thru 0x2266146 (0x6f18 bytes) 2024-09-14T07:00:11.911 INFO:teuthology.orchestra.run.smithi007.stdout:2653 writesame 0x7306a01 thru 0x730a1cb data_size 0x1b(0x37cb bytes) 2024-09-14T07:00:11.919 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:11.925 INFO:teuthology.orchestra.run.smithi007.stdout:2654 write 0xab7c93e thru 0xab861e5 (0x98a8 bytes) 2024-09-14T07:00:11.944 INFO:teuthology.orchestra.run.smithi007.stdout:2655 read 0xaa8275d thru 0xaa8f581 (0xce25 bytes) 2024-09-14T07:00:11.945 INFO:teuthology.orchestra.run.smithi007.stdout:2656 punch from 0x1457bf9 to 0x145dfad, (0x63b4 bytes) 2024-09-14T07:00:11.945 INFO:teuthology.orchestra.run.smithi007.stdout:2657 punch from 0x8ff29fb to 0x8ffd3f0, (0xa9f5 bytes) 2024-09-14T07:00:11.946 INFO:teuthology.orchestra.run.smithi007.stdout:2658 read 0x709f1f7 thru 0x70a19d9 (0x27e3 bytes) 2024-09-14T07:00:11.946 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:11.946 INFO:teuthology.orchestra.run.smithi007.stdout:2662 read 0x3f30854 thru 0x3f38057 (0x7804 bytes) 2024-09-14T07:00:11.947 INFO:teuthology.orchestra.run.smithi007.stdout:2663 read 0xff4482 thru 0xfff146 (0xacc5 bytes) 2024-09-14T07:00:11.953 INFO:teuthology.orchestra.run.smithi007.stdout:2664 write 0xe3013d7 thru 0xe30c512 (0xb13c bytes) 2024-09-14T07:00:11.954 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:11.952+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:11.967 INFO:teuthology.orchestra.run.smithi007.stdout:2666 write 0x5c29783 thru 0x5c37df8 (0xe676 bytes) 2024-09-14T07:00:11.983 INFO:teuthology.orchestra.run.smithi007.stdout:2667 write 0x89a15c8 thru 0x89a9909 (0x8342 bytes) 2024-09-14T07:00:11.998 INFO:teuthology.orchestra.run.smithi007.stdout:2668 writesame 0x71f6b62 thru 0x71fcae1 data_size 0x40(0x5f80 bytes) 2024-09-14T07:00:12.010 INFO:teuthology.orchestra.run.smithi007.stdout:2673 writesame 0xec70eb3 thru 0xec75dfa data_size 0x2b(0x4f48 bytes) 2024-09-14T07:00:12.017 INFO:teuthology.orchestra.run.smithi007.stdout:2675 trunc from 0xec75dfb to 0x8665ee5 2024-09-14T07:00:12.044 INFO:teuthology.orchestra.run.smithi007.stdout:2676 writesame 0x3f76fdf thru 0x3f857bc data_size 0x1(0xe7de bytes) 2024-09-14T07:00:12.054 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:12.052+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:12.060 INFO:teuthology.orchestra.run.smithi007.stdout:2677 read 0xd3abf0 thru 0xd3b2dd (0x6ee bytes) 2024-09-14T07:00:12.061 INFO:teuthology.orchestra.run.smithi007.stdout:2681 read 0x406bc56 thru 0x4070dcf (0x517a bytes) 2024-09-14T07:00:12.067 INFO:teuthology.orchestra.run.smithi007.stdout:2683 trunc from 0x8665ee5 to 0xe31db14 2024-09-14T07:00:12.078 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:12.079 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:12.079 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:12.079 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:12.079 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:12.080 INFO:teuthology.orchestra.run.smithi007.stdout:2684 trunc from 0xe31db14 to 0x7b2c4f2 2024-09-14T07:00:12.088 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:12.094 INFO:teuthology.orchestra.run.smithi007.stdout:2685 writesame 0xbab685c thru 0xbab8c3e data_size 0x1(0x23e3 bytes) 2024-09-14T07:00:12.094 INFO:teuthology.orchestra.run.smithi007.stdout:2686 punch from 0x1b9d595 to 0x1ba5b54, (0x85bf bytes) 2024-09-14T07:00:12.094 INFO:teuthology.orchestra.run.smithi007.stdout:2687 read 0xab002c4 thru 0xab0f1f5 (0xef32 bytes) 2024-09-14T07:00:12.097 INFO:teuthology.orchestra.run.smithi007.stdout:2688 trunc from 0xbab8c3f to 0xe6fed68 2024-09-14T07:00:12.102 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:12.102 INFO:teuthology.orchestra.run.smithi007.stdout:2690 write 0x11f30b3 thru 0x11fb836 (0x8784 bytes) 2024-09-14T07:00:12.118 INFO:teuthology.orchestra.run.smithi007.stdout:2692 read 0x302d690 thru 0x3039c48 (0xc5b9 bytes) 2024-09-14T07:00:12.118 INFO:teuthology.orchestra.run.smithi007.stdout:2695 write 0xa5750ac thru 0xa57c119 (0x706e bytes) 2024-09-14T07:00:12.132 INFO:teuthology.orchestra.run.smithi007.stdout:2697 writesame 0x3d7fd72 thru 0x3d82129 data_size 0x24(0x23b8 bytes) 2024-09-14T07:00:12.145 INFO:teuthology.orchestra.run.smithi007.stdout:2698 write 0x95e98cf thru 0x95f1f09 (0x863b bytes) 2024-09-14T07:00:12.155 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:12.154+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:12.160 INFO:teuthology.orchestra.run.smithi007.stdout:2699 read 0xa1a3cd3 thru 0xa1b22c5 (0xe5f3 bytes) 2024-09-14T07:00:12.170 INFO:teuthology.orchestra.run.smithi007.stdout:2700 trunc from 0xe6fed68 to 0x61837f1 2024-09-14T07:00:12.189 INFO:teuthology.orchestra.run.smithi007.stdout:2701 punch from 0x495a66c to 0x4960234, (0x5bc8 bytes) 2024-09-14T07:00:12.189 INFO:teuthology.orchestra.run.smithi007.stdout:2702 read 0x5621c29 thru 0x562e1d6 (0xc5ae bytes) 2024-09-14T07:00:12.189 INFO:teuthology.orchestra.run.smithi007.stdout:2703 read 0x7e015c thru 0x7e6c83 (0x6b28 bytes) 2024-09-14T07:00:12.191 INFO:teuthology.orchestra.run.smithi007.stdout:2704 read 0x31a705b thru 0x31ad309 (0x62af bytes) 2024-09-14T07:00:12.201 INFO:teuthology.orchestra.run.smithi007.stdout:2706 writesame 0x94726ec thru 0x94765ed data_size 0x5(0x3f02 bytes) 2024-09-14T07:00:12.201 INFO:teuthology.orchestra.run.smithi007.stdout:2709 read 0x27ede2d thru 0x27fd410 (0xf5e4 bytes) 2024-09-14T07:00:12.202 INFO:teuthology.orchestra.run.smithi007.stdout:2710 write 0x6b55d13 thru 0x6b5cb3c (0x6e2a bytes) 2024-09-14T07:00:12.226 INFO:teuthology.orchestra.run.smithi007.stdout:2712 write 0xcff46fa thru 0xcff88d1 (0x41d8 bytes) 2024-09-14T07:00:12.246 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:12.246 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:12.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:12.247 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:12.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.247 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:00:12.247 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:12.247 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.480446381, 2024-09-14T07:00:12.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:00:12.248 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.248 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.248 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.248 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:12.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:12.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:00:12.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:00:12.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:00:12.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:00:12.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:00:12.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:00:12.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:00:12.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:00:12.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:00:12.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:00:12.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:00:12.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:00:12.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:00:12.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:00:12.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:00:12.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:00:12.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.256 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.257 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:12.257 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:12.257 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.407327115000001, 2024-09-14T07:00:12.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:00:12.258 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.258 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.258 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.259 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:00:12.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.259 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:12.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:12.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:00:12.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:00:12.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:00:12.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:00:12.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:00:12.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:00:12.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:00:12.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:00:12.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:00:12.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:00:12.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:00:12.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:00:12.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:00:12.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:00:12.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:00:12.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:12.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:00:12.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:00:12.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:12.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:00:12.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:00:12.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:00:12.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:00:12.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:00:12.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:00:12.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.269 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.270 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:12.270 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:12.270 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.407070462, 2024-09-14T07:00:12.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:00:12.270 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.270 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.270 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.271 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.271 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.271 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.271 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:00:12.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.271 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:12.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:12.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:00:12.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:00:12.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:00:12.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:00:12.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:00:12.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:00:12.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:00:12.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:00:12.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:00:12.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:00:12.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:00:12.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:00:12.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:00:12.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:00:12.279 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.280 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.280 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:12.281 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:12.281 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.380783387999998, 2024-09-14T07:00:12.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:00:12.281 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.282 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.282 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.282 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.282 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.283 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.283 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:00:12.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.283 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:12.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:12.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:00:12.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:00:12.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:00:12.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:00:12.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:00:12.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:00:12.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:00:12.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:00:12.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:00:12.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.292 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.380658773999997, 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.293 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:00:12.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:12.297 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.886039611999998, 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:12.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:00:12.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.301 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.611636952000001, 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:00:12.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.617021342000001, 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:00:12.306 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:00:12.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:00:12.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.426848876000001, 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:00:12.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.377715586000001, 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.315 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:00:12.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:00:12.319 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.009496622, 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:00:12.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.888927682999999, 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.324 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.758011801999999, 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:00:12.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:00:12.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.728974539999999, 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.333 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:00:12.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.336 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.337 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.703205118, 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:12.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.669846974999999, 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.342 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:00:12.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:12.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:00:12.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:00:12.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:00:12.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.505639194, 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.347 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:00:12.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.494259933999999, 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:00:12.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:00:12.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.49421224, 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.356 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:12.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:00:12.360 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 27.493869915000001, 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:12.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.364 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.365 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.365 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.365 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.365 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:12.367 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:12.367 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:12.365+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2714 writesame 0xe55c671 thru 0xe567f9b data_size 0x17(0xb92b bytes) 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2716 writesame 0xa8bf7ee thru 0xa8cd22d data_size 0x61(0xda40 bytes) 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2717 punch from 0x79f4ec8 to 0x79fd6da, (0x8812 bytes) 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2718 write 0xed9e673 thru 0xeda232c (0x3cba bytes) 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2720 write 0x9dd036 thru 0x9dd1d7 (0x1a2 bytes) 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2721 read 0xec50dd thru 0xecc8bd (0x77e1 bytes) 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2723 write 0x8cf8350 thru 0x8cfccf6 (0x49a7 bytes) 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2724 read 0x581d3ca thru 0x582c4b8 (0xf0ef bytes) 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2725 read 0xe783648 thru 0xe78db22 (0xa4db bytes) 2024-09-14T07:00:12.368 INFO:teuthology.orchestra.run.smithi007.stdout:2728 writesame 0x2679986 thru 0x267d52f data_size 0x7(0x3baa bytes) 2024-09-14T07:00:12.369 INFO:teuthology.orchestra.run.smithi007.stdout:2729 clone 19 order 22 su 524288 sc 4 2024-09-14T07:00:12.443 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T07:00:12.467 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:12.465+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:12.499 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone18 from 0xeda232d (overlap 0x7583b6) to 0x524299 2024-09-14T07:00:12.517 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:12.517 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:12.517 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:12.526 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:12.568 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:12.566+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:12.668 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:12.668 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:12.668 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:12.668 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:12.668 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:12.668 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:12.668+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:12.676 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:12.769 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:12.769+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:12.780 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:12.780 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:12.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:12.780 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8489 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8208384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108897601999999, 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033127139, 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8489 2024-09-14T07:00:12.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670276+0000", 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670277+0000", 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6109999999999999e-06 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670283+0000", 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7409999999999998e-06 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670287+0000", 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1679999999999997e-06 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703311+0000", 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033023640999999999 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703352+0000", 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1168999999999998e-05 2024-09-14T07:00:12.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703402+0000", 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0424999999999999e-05 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8490 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108880019000001, 2024-09-14T07:00:12.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033303263, 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8490 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670295+0000", 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670300+0000", 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5369999999999997e-06 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670304+0000", 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8789999999999996e-06 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703433+0000", 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033128697999999998 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703510+0000", 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6662000000000006e-05 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703596+0000", 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6614999999999997e-05 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.789 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8491 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8454144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108862388999999, 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033389611999999999, 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.790 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8491 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4900000000000001e-07 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670317+0000", 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.19e-06 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670323+0000", 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:12.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670331+0000", 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1280000000000008e-06 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703611+0000", 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033279557000000001 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703650+0000", 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9410999999999999e-05 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703700+0000", 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0117000000000001e-05 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8492 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8568832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.794 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108834339000001, 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033561793999999999, 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8492 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670341+0000", 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8700000000000001e-06 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670347+0000", 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000004e-06 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670351+0000", 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4660000000000001e-06 2024-09-14T07:00:12.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703731+0000", 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033379502999999998 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703812+0000", 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1192000000000005e-05 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703900+0000", 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8382999999999997e-05 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8493 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8683520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108815840999998, 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033663516999999997, 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.799 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8493 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:12.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670359+0000", 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.685e-06 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670365+0000", 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.57e-06 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670369+0000", 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4460000000000003e-06 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703928+0000", 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033558530000000003 2024-09-14T07:00:12.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703970+0000", 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1909999999999997e-05 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704021+0000", 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0988e-05 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8494 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8798208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108786245000001, 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033828579999999997, 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.804 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8494 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670389+0000", 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.64e-06 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670393+0000", 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3470000000000001e-06 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670396+0000", 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6369999999999999e-06 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704051+0000", 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033655293000000003 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704128+0000", 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7453000000000005e-05 2024-09-14T07:00:12.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704215+0000", 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6884000000000004e-05 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8495 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8912896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108772694999999, 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033930725000000002, 2024-09-14T07:00:12.808 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8495 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670401+0000", 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.15e-07 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670402+0000", 2024-09-14T07:00:12.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.850000000000001e-07 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670405+0000", 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.225e-06 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670407+0000", 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.34e-06 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704240+0000", 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033833019999999998 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704281+0000", 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1332999999999999e-05 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704331+0000", 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9506999999999999e-05 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.812 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8496 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9027584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108761371, 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034114617, 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.813 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8496 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670413+0000", 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8999999999999995e-07 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670416+0000", 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2729999999999998e-06 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670418+0000", 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3709999999999998e-06 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704362+0000", 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033943917999999997 2024-09-14T07:00:12.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704439+0000", 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6922999999999995e-05 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704526+0000", 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7035999999999998e-05 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8497 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9142272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108751482999999, 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034220315000000001, 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8497 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670422+0000", 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:12.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670426+0000", 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5029999999999999e-06 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670430+0000", 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-06 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670432+0000", 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:00:12.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704552+0000", 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034120277999999997 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704592+0000", 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9884e-05 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704642+0000", 2024-09-14T07:00:12.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9623999999999998e-05 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8498 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9256960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108737720000001, 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034407434000000001, 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.822 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8498 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670436+0000", 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:00:12.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670439+0000", 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670442+0000", 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3120000000000001e-06 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704673+0000", 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034231175000000003 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704749+0000", 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6069999999999995e-05 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704843+0000", 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3622000000000006e-05 2024-09-14T07:00:12.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.827 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.827 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.827 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8499 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9371648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.827 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:12.827 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108728200000002, 2024-09-14T07:00:12.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034523731000000002, 2024-09-14T07:00:12.827 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8499 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:12.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2000000000000001e-07 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670446+0000", 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670449+0000", 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809999999999998e-06 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670455+0000", 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7930000000000003e-06 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704869+0000", 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034413931000000002 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704910+0000", 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1032999999999998e-05 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704968+0000", 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8897999999999998e-05 2024-09-14T07:00:12.831 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8500 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9486336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108714923000001, 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034708731, 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.832 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8500 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3300000000000001e-07 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670459+0000", 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670462+0000", 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1350000000000001e-06 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670470+0000", 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7400000000000004e-06 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705002+0000", 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034531470000000002 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705079+0000", 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7472999999999995e-05 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705167+0000", 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7750000000000005e-05 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8501 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9601024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108696494, 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034806278000000003, 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.836 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8501 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670477+0000", 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.44e-07 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670479+0000", 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7740000000000001e-06 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670485+0000", 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9150000000000001e-06 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670489+0000", 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5469999999999998e-06 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705191+0000", 2024-09-14T07:00:12.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034701896000000003 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705234+0000", 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2394000000000001e-05 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705283+0000", 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9208e-05 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8502 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9715712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108676155000001, 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034979270999999999, 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.841 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8502 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670506+0000", 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9740000000000008e-06 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670512+0000", 2024-09-14T07:00:12.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670517+0000", 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7049999999999998e-06 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705313+0000", 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034795858999999998 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705391+0000", 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8166999999999999e-05 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705476+0000", 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5285999999999996e-05 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.845 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8503 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9830400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.846 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:12.846 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108649733, 2024-09-14T07:00:12.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053094, 2024-09-14T07:00:12.846 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.846 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.846 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.846 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.846 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.846 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8503 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670524+0000", 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670525+0000", 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.581e-06 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670531+0000", 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8969999999999996e-06 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670536+0000", 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4630000000000003e-06 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705488+0000", 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034952451000000002 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705526+0000", 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8396999999999998e-05 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705576+0000", 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9926000000000003e-05 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8504 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9945088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.851 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108631395, 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035228601999999998, 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8504 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670543+0000", 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5820000000000001e-06 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670549+0000", 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670554+0000", 2024-09-14T07:00:12.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3699999999999997e-06 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705607+0000", 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053594 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705685+0000", 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8102999999999994e-05 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705770+0000", 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4678000000000005e-05 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8505 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10059776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108613607999999, 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035318548999999998, 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.856 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8505 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670560+0000", 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670561+0000", 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670568+0000", 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9789999999999996e-06 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670572+0000", 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2509999999999996e-06 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705784+0000", 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035211724 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705827+0000", 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3276999999999999e-05 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705878+0000", 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0488000000000002e-05 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8506 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10174464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108593522, 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035495512999999999, 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.861 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8506 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670580+0000", 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670582+0000", 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670587+0000", 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.767e-06 2024-09-14T07:00:12.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670592+0000", 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2819999999999998e-06 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705909+0000", 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.03531753 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705987+0000", 2024-09-14T07:00:12.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7787999999999995e-05 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706075+0000", 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7998999999999996e-05 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8507 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10289152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108575695999999, 2024-09-14T07:00:12.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035582134000000001, 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8507 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670598+0000", 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9900000000000002e-07 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670599+0000", 2024-09-14T07:00:12.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6029999999999999e-06 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670609+0000", 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.004e-05 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670614+0000", 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0749999999999997e-06 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706089+0000", 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035474358999999997 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706129+0000", 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0204999999999999e-05 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706179+0000", 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0553000000000002e-05 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.869 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8508 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10403840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.108552067000002, 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035743559000000001, 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8508 2024-09-14T07:00:12.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:12.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2800000000000002e-07 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670623+0000", 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670629+0000", 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.02e-06 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670634+0000", 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6280000000000004e-06 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706200+0000", 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035566564000000002 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706276+0000", 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5801000000000002e-05 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:12.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:12.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:12.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706364+0000", 2024-09-14T07:00:12.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8176000000000004e-05 2024-09-14T07:00:12.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.874 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:12.874 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:12.874 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:12.875 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:12.874+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:12.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:12.976 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:12.974+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:12.991 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:12.991 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:13.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:13.077 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:13.076+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:13.107 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:13.107 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:13.108 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:13.108 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:13.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:13.178 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:13.178+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:13.225 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:13.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.819447502999999, 2024-09-14T07:00:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:00:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:00:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:00:13.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.819385663, 2024-09-14T07:00:13.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:00:13.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:00:13.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.819352487, 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:00:13.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:00:13.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.819229615999999, 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.819194867, 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:13.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.251 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.819170938999999, 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:00:13.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.819148705, 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.257 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:00:13.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.819115737000001, 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:00:13.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:13.266 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.818764966, 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:00:13.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.818660783, 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.272 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:13.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:00:13.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:00:13.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.818620273000001, 2024-09-14T07:00:13.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:00:13.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:00:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:00:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:00:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:00:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.818595153, 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:00:13.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:13.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:00:13.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:00:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:00:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:00:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:00:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:00:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:00:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:00:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.818440856, 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:00:13.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:00:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:00:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:00:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:00:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.295 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.818407860000001, 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.296 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:13.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:00:13.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.818384954999999, 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:13.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:00:13.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:00:13.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.818334952000001, 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:00:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:00:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:00:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:00:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:00:13.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.6236028999999998, 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.314 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:00:13.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:00:13.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.6188963859999999, 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:00:13.318 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:00:13.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.615512721, 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:13.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:00:13.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:13.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:00:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:00:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:00:13.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.615492986, 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:13.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:00:13.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:00:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:00:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.333 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.333 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:13.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:13.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:13.336 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:13.334+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:13.436 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:13.435+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:13.518 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:13.518 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:13.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:118594 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=44b,call rbd.object_map_update in=39b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.515818+0000", 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.00085941499999999998, 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087748100000000005, 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for sub ops", 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 118594 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:13.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.515818+0000", 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.515818+0000", 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.515819+0000", 2024-09-14T07:00:13.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4139999999999999e-06 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.515825+0000", 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7429999999999999e-06 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.515853+0000", 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.794e-05 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:13.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.515861+0000", 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9289999999999998e-06 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.515911+0000", 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.515968+0000", 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6471999999999997e-05 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:13.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.516179+0000", 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021175100000000001 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-14T07:00:13.523 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:13.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:13.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:13.553 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:13.554 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:13.553+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:13.598 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:13.631 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:13.631 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:13.638 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:13.637+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:13.664 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:13.665 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:13.697 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:13.697 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:13.716 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:13.716 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:13.729 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:13.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:13.757 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:13.758 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:13.758 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:13.757+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:13.784 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:13.784 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:13.840 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:13.839+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:13.842 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-14T07:00:13.886 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:13.886 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:13.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:13.941 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:13.940+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:14.013 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:14.013 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:14.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:14.042 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:14.041+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:14.091 INFO:teuthology.orchestra.run.smithi007.stdout:2731 read 0xcae1dc0 thru 0xcaf05b0 (0xe7f1 bytes) 2024-09-14T07:00:14.101 INFO:teuthology.orchestra.run.smithi007.stdout:2732 trunc from 0xeda232d to 0x6c53764 2024-09-14T07:00:14.139 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:14.139 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:14.140 INFO:teuthology.orchestra.run.smithi007.stdout:2733 writesame 0xbc04db6 thru 0xbc06d3c data_size 0x1(0x1f87 bytes) 2024-09-14T07:00:14.140 INFO:teuthology.orchestra.run.smithi007.stdout:2736 trunc from 0xbc06d3d to 0xbca2eb5 2024-09-14T07:00:14.142 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:14.142+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:14.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:14.151 INFO:teuthology.orchestra.run.smithi007.stdout:2737 trunc from 0xbca2eb5 to 0x677b7cb 2024-09-14T07:00:14.157 INFO:teuthology.orchestra.run.smithi007.stdout:2738 read 0x3dd8517 thru 0x3de230d (0x9df7 bytes) 2024-09-14T07:00:14.158 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:14.158 INFO:teuthology.orchestra.run.smithi007.stdout:2740 punch from 0x3329dee to 0x332f917, (0x5b29 bytes) 2024-09-14T07:00:14.158 INFO:teuthology.orchestra.run.smithi007.stdout:2741 read 0x5da508d thru 0x5da962c (0x45a0 bytes) 2024-09-14T07:00:14.159 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:14.159 INFO:teuthology.orchestra.run.smithi007.stdout:2743 read 0x3207107 thru 0x32163a0 (0xf29a bytes) 2024-09-14T07:00:14.160 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:14.160 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:14.160 INFO:teuthology.orchestra.run.smithi007.stdout:2746 read 0x2a2c7e2 thru 0x2a2e4b8 (0x1cd7 bytes) 2024-09-14T07:00:14.160 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:14.160 INFO:teuthology.orchestra.run.smithi007.stdout:2749 write 0x4e66f8c thru 0x4e7694e (0xf9c3 bytes) 2024-09-14T07:00:14.166 INFO:teuthology.orchestra.run.smithi007.stdout:2750 read 0xf8f8e5 thru 0xf9e1c2 (0xe8de bytes) 2024-09-14T07:00:14.166 INFO:teuthology.orchestra.run.smithi007.stdout:2752 write 0x8cd9d9 thru 0x8d247f (0x4aa7 bytes) 2024-09-14T07:00:14.244 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:14.243+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:14.282 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:14.282 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:14.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:14.306 INFO:teuthology.orchestra.run.smithi007.stdout:2755 write 0x6ee1dd7 thru 0x6ee5e0c (0x4036 bytes) 2024-09-14T07:00:14.308 INFO:teuthology.orchestra.run.smithi007.stdout:2756 read 0x1a086a3 thru 0x1a10b43 (0x84a1 bytes) 2024-09-14T07:00:14.308 INFO:teuthology.orchestra.run.smithi007.stdout:2757 read 0x133b530 thru 0x134222b (0x6cfc bytes) 2024-09-14T07:00:14.309 INFO:teuthology.orchestra.run.smithi007.stdout:2758 read 0x55ebeb2 thru 0x55f8d63 (0xceb2 bytes) 2024-09-14T07:00:14.309 INFO:teuthology.orchestra.run.smithi007.stdout:2759 read 0x428f93d thru 0x42916ed (0x1db1 bytes) 2024-09-14T07:00:14.319 INFO:teuthology.orchestra.run.smithi007.stdout:2760 write 0xb8e24a4 thru 0xb8ec51d (0xa07a bytes) 2024-09-14T07:00:14.323 INFO:teuthology.orchestra.run.smithi007.stdout:2761 read 0xb1d57a3 thru 0xb1dc7df (0x703d bytes) 2024-09-14T07:00:14.323 INFO:teuthology.orchestra.run.smithi007.stdout:2763 writesame 0x5013e72 thru 0x5014082 data_size 0x1(0x211 bytes) 2024-09-14T07:00:14.335 INFO:teuthology.orchestra.run.smithi007.stdout:2764 writesame 0xbad7660 thru 0xbae12b4 data_size 0x1(0x9c55 bytes) 2024-09-14T07:00:14.336 INFO:teuthology.orchestra.run.smithi007.stdout:2765 read 0xa733cb8 thru 0xa738f94 (0x52dd bytes) 2024-09-14T07:00:14.336 INFO:teuthology.orchestra.run.smithi007.stdout:2766 read 0x23c6ef3 thru 0x23cfd3e (0x8e4c bytes) 2024-09-14T07:00:14.337 INFO:teuthology.orchestra.run.smithi007.stdout:2767 write 0x77d7c64 thru 0x77d8fbc (0x1359 bytes) 2024-09-14T07:00:14.345 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:14.343+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:14.348 INFO:teuthology.orchestra.run.smithi007.stdout:2768 trunc from 0xbae12b5 to 0x46497f1 2024-09-14T07:00:14.367 INFO:teuthology.orchestra.run.smithi007.stdout:2769 writesame 0x1ff99b1 thru 0x20088a0 data_size 0x8(0xeef0 bytes) 2024-09-14T07:00:14.384 INFO:teuthology.orchestra.run.smithi007.stdout:2774 read 0x2c4fec2 thru 0x2c5ddd4 (0xdf13 bytes) 2024-09-14T07:00:14.384 INFO:teuthology.orchestra.run.smithi007.stdout:2776 clone 20 order 22 su 2097152 sc 8 2024-09-14T07:00:14.426 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:14.426 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:14.434 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:14.445 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:14.445 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:14.445 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 7] 2024-09-14T07:00:14.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-14T07:00:14.446 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:14.445+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:14.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:00:14.519 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone19 from 0x46497f1 (overlap 0x46497f1) to 0x707598 2024-09-14T07:00:14.560 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:14.559+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:14.649 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:14.648+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:14.651 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:14.651 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:14.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:14.750 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:14.748+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:14.850 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:14.850+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:14.895 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-14T07:00:14.903 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:14.903 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:14.918 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:14.960 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:14.959+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:15.047 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-14T07:00:15.046+0000 7fbb91e53640 -1 mon.a@0(leader).osd e62 definitely_dead 0 2024-09-14T07:00:15.051 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:15.052 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:15.052 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:15.050+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:15.062 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:15.141 INFO:teuthology.orchestra.run.smithi007.stdout:2778 write 0xdec69da thru 0xdecba7b (0x50a2 bytes) 2024-09-14T07:00:15.144 INFO:teuthology.orchestra.run.smithi007.stdout:2779 write 0xb96765d thru 0xb96b878 (0x421c bytes) 2024-09-14T07:00:15.147 INFO:teuthology.orchestra.run.smithi007.stdout:2781 writesame 0xb596333 thru 0xb5972b5 data_size 0xb(0xf83 bytes) 2024-09-14T07:00:15.150 INFO:teuthology.orchestra.run.smithi007.stdout:2782 read 0x31bf57b thru 0x31c2aad (0x3533 bytes) 2024-09-14T07:00:15.150 INFO:teuthology.orchestra.run.smithi007.stdout:2784 writesame 0xde78762 thru 0xde81db9 data_size 0x44(0x9658 bytes) 2024-09-14T07:00:15.152 INFO:teuthology.orchestra.run.smithi007.stdout:2785 writesame 0xba0521a thru 0xba14d35 data_size 0x42(0xfb1c bytes) 2024-09-14T07:00:15.152 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:15.151+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:15.155 INFO:teuthology.orchestra.run.smithi007.stdout:2787 read 0x2f9146e thru 0x2f9f417 (0xdfaa bytes) 2024-09-14T07:00:15.156 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.168 INFO:teuthology.orchestra.run.smithi007.stdout:2790 trunc from 0xdecba7c to 0x380ad06 2024-09-14T07:00:15.190 INFO:teuthology.orchestra.run.smithi007.stdout:2791 trunc from 0x380ad06 to 0x16b541c 2024-09-14T07:00:15.216 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:15.216 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:15.221 INFO:teuthology.orchestra.run.smithi007.stdout:2792 read 0x50a152 thru 0x50ab49 (0x9f8 bytes) 2024-09-14T07:00:15.225 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:15.234 INFO:teuthology.orchestra.run.smithi007.stdout:2793 trunc from 0x16b541c to 0xc403c5f 2024-09-14T07:00:15.239 INFO:teuthology.orchestra.run.smithi007.stdout:2794 punch from 0xafcfbbf to 0xafd828c, (0x86cd bytes) 2024-09-14T07:00:15.239 INFO:teuthology.orchestra.run.smithi007.stdout:2796 read 0x8a45046 thru 0x8a460df (0x109a bytes) 2024-09-14T07:00:15.239 INFO:teuthology.orchestra.run.smithi007.stdout:2797 writesame 0x5be6748 thru 0x5bee1e0 data_size 0x1(0x7a99 bytes) 2024-09-14T07:00:15.256 INFO:teuthology.orchestra.run.smithi007.stdout:2798 writesame 0x8c7f4ba thru 0x8c8f11a data_size 0x1(0xfc61 bytes) 2024-09-14T07:00:15.267 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:15.265+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:15.286 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.286 INFO:teuthology.orchestra.run.smithi007.stdout:2801 read 0x246a7d3 thru 0x247177e (0x6fac bytes) 2024-09-14T07:00:15.286 INFO:teuthology.orchestra.run.smithi007.stdout:2802 read 0xa71f47b thru 0xa72d8c0 (0xe446 bytes) 2024-09-14T07:00:15.287 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.287 INFO:teuthology.orchestra.run.smithi007.stdout:2804 read 0x1d58857 thru 0x1d5a4b7 (0x1c61 bytes) 2024-09-14T07:00:15.287 INFO:teuthology.orchestra.run.smithi007.stdout:2805 read 0x8d4beff thru 0x8d57424 (0xb526 bytes) 2024-09-14T07:00:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:2806 trunc from 0xc403c5f to 0x5e7c175 2024-09-14T07:00:15.309 INFO:teuthology.orchestra.run.smithi007.stdout:2807 write 0x5fc5cb0 thru 0x5fd0b49 (0xae9a bytes) 2024-09-14T07:00:15.311 INFO:teuthology.orchestra.run.smithi007.stdout:2809 writesame 0x5bdb930 thru 0x5be23d9 data_size 0x52(0x6aaa bytes) 2024-09-14T07:00:15.313 INFO:teuthology.orchestra.run.smithi007.stdout:2810 write 0x5cb306d thru 0x5cc0572 (0xd506 bytes) 2024-09-14T07:00:15.316 INFO:teuthology.orchestra.run.smithi007.stdout:2811 read 0x7fe84c thru 0x80d50b (0xecc0 bytes) 2024-09-14T07:00:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:2816 punch from 0x4a00df to 0x4af11f, (0xf040 bytes) 2024-09-14T07:00:15.317 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:2818 writesame 0x8597fde thru 0x859c6a8 data_size 0x7(0x46cb bytes) 2024-09-14T07:00:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:2821 punch from 0x323cf80 to 0x323d27a, (0x2fa bytes) 2024-09-14T07:00:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:2823 write 0x2e00764 thru 0x2e0f568 (0xee05 bytes) 2024-09-14T07:00:15.328 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.335 INFO:teuthology.orchestra.run.smithi007.stdout:2825 trunc from 0x859c6a9 to 0x206c36b 2024-09-14T07:00:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.355 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:15.353+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:2830 writesame 0x924c1da thru 0x924fea7 data_size 0x2b(0x3cce bytes) 2024-09-14T07:00:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:2833 punch from 0x20d0932 to 0x20ddd8a, (0xd458 bytes) 2024-09-14T07:00:15.361 INFO:teuthology.orchestra.run.smithi007.stdout:2835 read 0x4a7eebb thru 0x4a88601 (0x9747 bytes) 2024-09-14T07:00:15.371 INFO:teuthology.orchestra.run.smithi007.stdout:2837 writesame 0xda35c68 thru 0xda448f6 data_size 0x1(0xec8f bytes) 2024-09-14T07:00:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:2840 punch from 0xaecd3c to 0xaf6199, (0x945d bytes) 2024-09-14T07:00:15.372 INFO:teuthology.orchestra.run.smithi007.stdout:2842 write 0x602eed4 thru 0x603bd6a (0xce97 bytes) 2024-09-14T07:00:15.373 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:15.373 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:15.374 INFO:teuthology.orchestra.run.smithi007.stdout:2843 punch from 0xd06f6a3 to 0xd07d5aa, (0xdf07 bytes) 2024-09-14T07:00:15.375 INFO:teuthology.orchestra.run.smithi007.stdout:2844 write 0xbb2a771 thru 0xbb35c00 (0xb490 bytes) 2024-09-14T07:00:15.378 INFO:teuthology.orchestra.run.smithi007.stdout:2845 write 0x83ef28b thru 0x83f42ef (0x5065 bytes) 2024-09-14T07:00:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:15.381 INFO:teuthology.orchestra.run.smithi007.stdout:2848 writesame 0x24e56f0 thru 0x24e7d60 data_size 0x1(0x2671 bytes) 2024-09-14T07:00:15.382 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:15.401 INFO:teuthology.orchestra.run.smithi007.stdout:2849 trunc from 0xda448f7 to 0x1d284fb 2024-09-14T07:00:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:2850 trunc from 0x1d284fb to 0x37938d7 2024-09-14T07:00:15.427 INFO:teuthology.orchestra.run.smithi007.stdout:2851 write 0x6b0d67f thru 0x6b0f444 (0x1dc6 bytes) 2024-09-14T07:00:15.431 INFO:teuthology.orchestra.run.smithi007.stdout:2852 read 0x15e2216 thru 0x15e70fa (0x4ee5 bytes) 2024-09-14T07:00:15.431 INFO:teuthology.orchestra.run.smithi007.stdout:2853 read 0x4dd762 thru 0x4e1de3 (0x4682 bytes) 2024-09-14T07:00:15.432 INFO:teuthology.orchestra.run.smithi007.stdout:2854 punch from 0x3028d6 to 0x308910, (0x603a bytes) 2024-09-14T07:00:15.432 INFO:teuthology.orchestra.run.smithi007.stdout:2855 punch from 0x53b4dee to 0x53c452b, (0xf73d bytes) 2024-09-14T07:00:15.432 INFO:teuthology.orchestra.run.smithi007.stdout:2856 read 0x3c87ae9 thru 0x3c8d1c3 (0x56db bytes) 2024-09-14T07:00:15.432 INFO:teuthology.orchestra.run.smithi007.stdout:2857 read 0x3785611 thru 0x378ee0c (0x97fc bytes) 2024-09-14T07:00:15.444 INFO:teuthology.orchestra.run.smithi007.stdout:2858 write 0xc88e4ea thru 0xc8970b8 (0x8bcf bytes) 2024-09-14T07:00:15.447 INFO:teuthology.orchestra.run.smithi007.stdout:2859 read 0x33d5434 thru 0x33e4351 (0xef1e bytes) 2024-09-14T07:00:15.450 INFO:teuthology.orchestra.run.smithi007.stdout:2860 trunc from 0xc8970b9 to 0xa777de3 2024-09-14T07:00:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.2. 2024-09-14T07:00:15.460 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T07:00:15.467 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:15.466+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:15.474 INFO:teuthology.orchestra.run.smithi007.stdout:2861 write 0xc9955dd thru 0xc997069 (0x1a8d bytes) 2024-09-14T07:00:15.477 INFO:teuthology.orchestra.run.smithi007.stdout:2862 punch from 0x7c9e9c5 to 0x7c9f1ee, (0x829 bytes) 2024-09-14T07:00:15.477 INFO:teuthology.orchestra.run.smithi007.stdout:2863 read 0xa3bdb16 thru 0xa3c2920 (0x4e0b bytes) 2024-09-14T07:00:15.477 INFO:teuthology.orchestra.run.smithi007.stdout:2864 read 0x53daa3c thru 0x53e4d10 (0xa2d5 bytes) 2024-09-14T07:00:15.477 INFO:teuthology.orchestra.run.smithi007.stdout:2865 read 0x359b57e thru 0x359ffc7 (0x4a4a bytes) 2024-09-14T07:00:15.477 INFO:teuthology.orchestra.run.smithi007.stdout:2866 read 0x8caa87c thru 0x8cb0008 (0x578d bytes) 2024-09-14T07:00:15.477 INFO:teuthology.orchestra.run.smithi007.stdout:2867 write 0x67f7c65 thru 0x67f950b (0x18a7 bytes) 2024-09-14T07:00:15.480 INFO:teuthology.orchestra.run.smithi007.stdout:2868 write 0xa328be3 thru 0xa330efa (0x8318 bytes) 2024-09-14T07:00:15.532 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:15.532 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:15.540 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:15.557 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:15.556+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:15.658 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:15.658+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:15.684 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:15.684 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:15.693 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:15.760 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:15.759+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:15.797 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:15.797 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:15.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:15.806 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:15.861 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:15.859+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:15.908 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:15.908 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:15.917 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:15.924 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:15.924 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:15.941 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:15.957 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:15.957 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:15.973 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:15.990 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:16.001 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:16.000+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:16.008 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:16.026 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:16.044 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:16.044 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:16.062 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:16.061+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:16.065 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:16.066 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:16.066 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:16.066 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:16.066 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:16.074 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:16.163 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:16.162+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:16.219 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:16.219 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:16.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:16.219 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 55.453814530000002, 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:00:16.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:16.224 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.380695264000003, 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:00:16.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.380438611000002, 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:00:16.230 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:00:16.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.234 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.354151537, 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.235 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:00:16.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:16.239 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.354026922999999, 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:16.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:00:16.243 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 36.859407761, 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:00:16.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:00:16.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:16.248 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 35.585005101, 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:16.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:00:16.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.590389491000003, 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:16.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:00:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.400217025000003, 2024-09-14T07:00:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:00:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:00:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:00:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:00:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:00:16.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:00:16.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.351083735000003, 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.263 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:00:16.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.982864770999999, 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:00:16.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:00:16.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:00:16.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.862295832000001, 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:00:16.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:00:16.276 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.731379951000001, 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:00:16.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.702342688999998, 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.281 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:00:16.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.676573266999998, 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:00:16.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:00:16.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.643215124000001, 2024-09-14T07:00:16.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:00:16.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.479007342999999, 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.295 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:00:16.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.299 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.467628083000001, 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.467580388999998, 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.304 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:00:16.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:00:16.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 31.467238064, 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:00:16.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:00:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:00:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.313 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.313 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:16.314 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:16.315 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:16.313+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:16.416 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:16.415+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:16.418 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:16.418 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:16.418 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:16.418 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:16.427 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:16.468 INFO:teuthology.orchestra.run.smithi007.stdout:2869 read 0x73d4438 thru 0x73df2ef (0xaeb8 bytes) 2024-09-14T07:00:16.469 INFO:teuthology.orchestra.run.smithi007.stdout:2870 writesame 0x4453359 thru 0x445bef7 data_size 0x1(0x8b9f bytes) 2024-09-14T07:00:16.485 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:16.486 INFO:teuthology.orchestra.run.smithi007.stdout:2873 trunc from 0xc99706a to 0xc92c1d7 2024-09-14T07:00:16.493 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:16.493 INFO:teuthology.orchestra.run.smithi007.stdout:2876 read 0x703277d thru 0x7039097 (0x691b bytes) 2024-09-14T07:00:16.493 INFO:teuthology.orchestra.run.smithi007.stdout:2877 writesame 0x58b36 thru 0x67079 data_size 0xc(0xe544 bytes) 2024-09-14T07:00:16.516 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:16.514+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:16.544 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:16.544 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:16.545 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:16.553 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:16.618 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:16.617+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:16.700 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:16.700 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:16.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:16.700 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8489 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8208384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028393676, 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033127139, 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8489 2024-09-14T07:00:16.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:16.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670276+0000", 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670277+0000", 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6109999999999999e-06 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670283+0000", 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7409999999999998e-06 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670287+0000", 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1679999999999997e-06 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703311+0000", 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033023640999999999 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703352+0000", 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1168999999999998e-05 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703402+0000", 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0424999999999999e-05 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.705 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8490 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028376092999999, 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033303263, 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8490 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:16.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670295+0000", 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670300+0000", 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5369999999999997e-06 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670304+0000", 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8789999999999996e-06 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703433+0000", 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033128697999999998 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703510+0000", 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6662000000000006e-05 2024-09-14T07:00:16.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703596+0000", 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6614999999999997e-05 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8491 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8454144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028358463000004, 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033389611999999999, 2024-09-14T07:00:16.710 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8491 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4900000000000001e-07 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670317+0000", 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.19e-06 2024-09-14T07:00:16.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670323+0000", 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670331+0000", 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1280000000000008e-06 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703611+0000", 2024-09-14T07:00:16.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033279557000000001 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703650+0000", 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9410999999999999e-05 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703700+0000", 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0117000000000001e-05 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8492 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8568832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028330412999999, 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033561793999999999, 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.715 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8492 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670341+0000", 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8700000000000001e-06 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670347+0000", 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000004e-06 2024-09-14T07:00:16.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670351+0000", 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4660000000000001e-06 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703731+0000", 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033379502999999998 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703812+0000", 2024-09-14T07:00:16.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1192000000000005e-05 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703900+0000", 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8382999999999997e-05 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8493 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8683520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028311915000003, 2024-09-14T07:00:16.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033663516999999997, 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8493 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:16.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670359+0000", 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.685e-06 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670365+0000", 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.57e-06 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670369+0000", 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4460000000000003e-06 2024-09-14T07:00:16.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703928+0000", 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033558530000000003 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703970+0000", 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1909999999999997e-05 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704021+0000", 2024-09-14T07:00:16.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0988e-05 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8494 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8798208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028282318999999, 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033828579999999997, 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.724 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8494 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670389+0000", 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.64e-06 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670393+0000", 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3470000000000001e-06 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670396+0000", 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6369999999999999e-06 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704051+0000", 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033655293000000003 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704128+0000", 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7453000000000005e-05 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704215+0000", 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6884000000000004e-05 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.728 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8495 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8912896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028268769, 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033930725000000002, 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8495 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670401+0000", 2024-09-14T07:00:16.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.15e-07 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670402+0000", 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.850000000000001e-07 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670405+0000", 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.225e-06 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670407+0000", 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.34e-06 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704240+0000", 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033833019999999998 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704281+0000", 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1332999999999999e-05 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704331+0000", 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9506999999999999e-05 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8496 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9027584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028257445000001, 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034114617, 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.733 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8496 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:16.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670413+0000", 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8999999999999995e-07 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670416+0000", 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2729999999999998e-06 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670418+0000", 2024-09-14T07:00:16.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3709999999999998e-06 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704362+0000", 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033943917999999997 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704439+0000", 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6922999999999995e-05 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704526+0000", 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7035999999999998e-05 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8497 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9142272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028247557, 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034220315000000001, 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.739 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.740 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.740 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.740 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.740 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8497 2024-09-14T07:00:16.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.740 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670422+0000", 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670426+0000", 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5029999999999999e-06 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670430+0000", 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-06 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670432+0000", 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:00:16.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704552+0000", 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034120277999999997 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704592+0000", 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9884e-05 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704642+0000", 2024-09-14T07:00:16.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9623999999999998e-05 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8498 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9256960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028233794000002, 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034407434000000001, 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.744 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8498 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:16.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670436+0000", 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670439+0000", 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670442+0000", 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3120000000000001e-06 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704673+0000", 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034231175000000003 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704749+0000", 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6069999999999995e-05 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704843+0000", 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3622000000000006e-05 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.748 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8499 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9371648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028224274000003, 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034523731000000002, 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8499 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2000000000000001e-07 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670446+0000", 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670449+0000", 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809999999999998e-06 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670455+0000", 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7930000000000003e-06 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704869+0000", 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034413931000000002 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704910+0000", 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1032999999999998e-05 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704968+0000", 2024-09-14T07:00:16.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8897999999999998e-05 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8500 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9486336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028210997000002, 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034708731, 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.753 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8500 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3300000000000001e-07 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670459+0000", 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670462+0000", 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1350000000000001e-06 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670470+0000", 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7400000000000004e-06 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705002+0000", 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034531470000000002 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705079+0000", 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7472999999999995e-05 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705167+0000", 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7750000000000005e-05 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8501 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9601024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:16.757 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028192568000001, 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034806278000000003, 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8501 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670477+0000", 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.44e-07 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670479+0000", 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7740000000000001e-06 2024-09-14T07:00:16.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670485+0000", 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9150000000000001e-06 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670489+0000", 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5469999999999998e-06 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705191+0000", 2024-09-14T07:00:16.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034701896000000003 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705234+0000", 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2394000000000001e-05 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705283+0000", 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9208e-05 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8502 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9715712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028172228999999, 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034979270999999999, 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8502 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.762 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670506+0000", 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9740000000000008e-06 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670512+0000", 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670517+0000", 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7049999999999998e-06 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705313+0000", 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034795858999999998 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705391+0000", 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8166999999999999e-05 2024-09-14T07:00:16.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705476+0000", 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5285999999999996e-05 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8503 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9830400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028145807000001, 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053094, 2024-09-14T07:00:16.766 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8503 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670524+0000", 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670525+0000", 2024-09-14T07:00:16.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.581e-06 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670531+0000", 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8969999999999996e-06 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670536+0000", 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4630000000000003e-06 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705488+0000", 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034952451000000002 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705526+0000", 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8396999999999998e-05 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705576+0000", 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9926000000000003e-05 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.770 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8504 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9945088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028127468999998, 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035228601999999998, 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8504 2024-09-14T07:00:16.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670543+0000", 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5820000000000001e-06 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670549+0000", 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670554+0000", 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3699999999999997e-06 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705607+0000", 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053594 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705685+0000", 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8102999999999994e-05 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705770+0000", 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4678000000000005e-05 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8505 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10059776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028109682, 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035318548999999998, 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.775 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8505 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670560+0000", 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670561+0000", 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670568+0000", 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9789999999999996e-06 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670572+0000", 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2509999999999996e-06 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705784+0000", 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035211724 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705827+0000", 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3276999999999999e-05 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705878+0000", 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0488000000000002e-05 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8506 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10174464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.779 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028089596000001, 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035495512999999999, 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8506 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670580+0000", 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670582+0000", 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670587+0000", 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.767e-06 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670592+0000", 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2819999999999998e-06 2024-09-14T07:00:16.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705909+0000", 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.03531753 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705987+0000", 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7787999999999995e-05 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706075+0000", 2024-09-14T07:00:16.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7998999999999996e-05 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8507 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10289152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028071769999997, 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035582134000000001, 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.784 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8507 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670598+0000", 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9900000000000002e-07 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670599+0000", 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6029999999999999e-06 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670609+0000", 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.004e-05 2024-09-14T07:00:16.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670614+0000", 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0749999999999997e-06 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706089+0000", 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035474358999999997 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706129+0000", 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0204999999999999e-05 2024-09-14T07:00:16.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706179+0000", 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0553000000000002e-05 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8508 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10403840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.028048140999999, 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035743559000000001, 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:16.788 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8508 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2800000000000002e-07 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670623+0000", 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T07:00:16.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670629+0000", 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.02e-06 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670634+0000", 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6280000000000004e-06 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706200+0000", 2024-09-14T07:00:16.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035566564000000002 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706276+0000", 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5801000000000002e-05 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706364+0000", 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8176000000000004e-05 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.792 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:16.793 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:16.793 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:16.794 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:16.793+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:16.794 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:16.895 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:16.894+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:16.911 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:16.911 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:16.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:16.996 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:16.994+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:17.067 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:17.068 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:17.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:17.096 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:17.095+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:17.185 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.780070720000001, 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.78000888, 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:00:17.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779975704000002, 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:17.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.200 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779852833, 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:00:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779818083999999, 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:00:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:00:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:00:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:00:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:00:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:00:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:00:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:00:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779794156000001, 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:00:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:00:17.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:00:17.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:00:17.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:00:17.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:00:17.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:00:17.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:00:17.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:00:17.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:00:17.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:00:17.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.218 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:17.219 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779771921999998, 2024-09-14T07:00:17.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:00:17.219 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:17.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:00:17.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:00:17.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779738953999999, 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:00:17.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:00:17.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:00:17.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:00:17.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779388183000002, 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:00:17.229 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.231 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.231 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:17.235 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779284000000001, 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:00:17.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:17.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:00:17.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779243489999999, 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:00:17.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:00:17.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:00:17.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779218369999999, 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:00:17.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:00:17.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:00:17.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779064073000001, 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.251 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:00:17.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.252 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:17.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:17.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:00:17.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:00:17.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:00:17.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779031076999999, 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:00:17.258 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:00:17.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.263 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.779008172000001, 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:00:17.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.778958168999999, 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.269 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:00:17.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:00:17.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:00:17.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:00:17.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:00:17.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.584226117, 2024-09-14T07:00:17.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:00:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:00:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.275 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:17.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:17.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:00:17.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:00:17.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:00:17.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:00:17.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:00:17.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:00:17.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:00:17.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:00:17.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:00:17.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:00:17.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:00:17.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:00:17.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:00:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:00:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.281 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:17.282 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:17.282 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.5795196029999996, 2024-09-14T07:00:17.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:00:17.282 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:00:17.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.283 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:17.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:00:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:00:17.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:00:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:00:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:00:17.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:00:17.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:00:17.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:17.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:00:17.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:00:17.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:00:17.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:00:17.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.5761359380000002, 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:00:17.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:00:17.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.5761162029999998, 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:00:17.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.298 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:17.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:17.300 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:17.298+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:17.400 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:17.399+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:17.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:17.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:00:17.500 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:17.499+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:17.601 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:17.599+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:17.646 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:17.646 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:17.646 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:17.651 INFO:teuthology.orchestra.run.smithi007.stdout:2878 read 0x6ddb802 thru 0x6de0003 (0x4802 bytes) 2024-09-14T07:00:17.659 INFO:teuthology.orchestra.run.smithi007.stdout:2880 trunc from 0xc92c1d7 to 0x5e53d8f 2024-09-14T07:00:17.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:17.673 INFO:teuthology.orchestra.run.smithi007.stdout:2882 read 0x5dd72d4 thru 0x5ddc18f (0x4ebc bytes) 2024-09-14T07:00:17.687 INFO:teuthology.orchestra.run.smithi007.stdout:2883 write 0xace00fa thru 0xace36a6 (0x35ad bytes) 2024-09-14T07:00:17.691 INFO:teuthology.orchestra.run.smithi007.stdout:2884 write 0x8bc4aaa thru 0x8bd38d2 (0xee29 bytes) 2024-09-14T07:00:17.694 INFO:teuthology.orchestra.run.smithi007.stdout:2885 write 0x35838e3 thru 0x3586b06 (0x3224 bytes) 2024-09-14T07:00:17.698 INFO:teuthology.orchestra.run.smithi007.stdout:2886 read 0x749885f thru 0x74a00e3 (0x7885 bytes) 2024-09-14T07:00:17.702 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:17.700+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:17.710 INFO:teuthology.orchestra.run.smithi007.stdout:2887 write 0xe758490 thru 0xe75d633 (0x51a4 bytes) 2024-09-14T07:00:17.720 INFO:teuthology.orchestra.run.smithi007.stdout:2888 trunc from 0xe75d634 to 0x96764a6 2024-09-14T07:00:17.736 INFO:teuthology.orchestra.run.smithi007.stdout:2889 write 0x88ad896 thru 0x88b78a8 (0xa013 bytes) 2024-09-14T07:00:17.741 INFO:teuthology.orchestra.run.smithi007.stdout:2891 punch from 0x50d333d to 0x50e0a0a, (0xd6cd bytes) 2024-09-14T07:00:17.741 INFO:teuthology.orchestra.run.smithi007.stdout:2892 punch from 0x3bfc48e to 0x3c04f91, (0x8b03 bytes) 2024-09-14T07:00:17.741 INFO:teuthology.orchestra.run.smithi007.stdout:2893 read 0x3bbaec3 thru 0x3bbce90 (0x1fce bytes) 2024-09-14T07:00:17.741 INFO:teuthology.orchestra.run.smithi007.stdout:2896 writesame 0x34a9322 thru 0x34b70eb data_size 0x3(0xddca bytes) 2024-09-14T07:00:17.752 INFO:teuthology.orchestra.run.smithi007.stdout:2897 punch from 0x638fa6 to 0x64575c, (0xc7b6 bytes) 2024-09-14T07:00:17.752 INFO:teuthology.orchestra.run.smithi007.stdout:2899 read 0x3c4a60d thru 0x3c50e9d (0x6891 bytes) 2024-09-14T07:00:17.752 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:17.752 INFO:teuthology.orchestra.run.smithi007.stdout:2901 read 0x6938f84 thru 0x694167d (0x86fa bytes) 2024-09-14T07:00:17.762 INFO:teuthology.orchestra.run.smithi007.stdout:2903 write 0xbf23cb1 thru 0xbf309f7 (0xcd47 bytes) 2024-09-14T07:00:17.766 INFO:teuthology.orchestra.run.smithi007.stdout:2904 read 0x392ad79 thru 0x3939bc9 (0xee51 bytes) 2024-09-14T07:00:17.767 INFO:teuthology.orchestra.run.smithi007.stdout:2905 write 0x1c20f4a thru 0x1c307d8 (0xf88f bytes) 2024-09-14T07:00:17.769 INFO:teuthology.orchestra.run.smithi007.stdout:2906 read 0x385909f thru 0x3862dae (0x9d10 bytes) 2024-09-14T07:00:17.773 INFO:teuthology.orchestra.run.smithi007.stdout:2908 trunc from 0xbf309f8 to 0x7f8baf0 2024-09-14T07:00:17.784 INFO:teuthology.orchestra.run.smithi007.stdout:2909 read 0x55aeacb thru 0x55baf33 (0xc469 bytes) 2024-09-14T07:00:17.785 INFO:teuthology.orchestra.run.smithi007.stdout:2910 write 0x2cf8730 thru 0x2d00a52 (0x8323 bytes) 2024-09-14T07:00:17.788 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:17.788 INFO:teuthology.orchestra.run.smithi007.stdout:2913 punch from 0x46282ad to 0x4628b99, (0x8ec bytes) 2024-09-14T07:00:17.788 INFO:teuthology.orchestra.run.smithi007.stdout:2914 write 0x72a9428 thru 0x72b423d (0xae16 bytes) 2024-09-14T07:00:17.791 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:17.793 INFO:teuthology.orchestra.run.smithi007.stdout:2917 trunc from 0x7f8baf0 to 0x974225e 2024-09-14T07:00:17.799 INFO:teuthology.orchestra.run.smithi007.stdout:2918 read 0x1273749 thru 0x1281306 (0xdbbe bytes) 2024-09-14T07:00:17.800 INFO:teuthology.orchestra.run.smithi007.stdout:2919 writesame 0x36a0618 thru 0x36a1752 data_size 0x1(0x113b bytes) 2024-09-14T07:00:17.803 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:17.801+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:17.805 INFO:teuthology.orchestra.run.smithi007.stdout:2920 write 0x178ac8d thru 0x178cc4f (0x1fc3 bytes) 2024-09-14T07:00:17.815 INFO:teuthology.orchestra.run.smithi007.stdout:2922 writesame 0xc3cf532 thru 0xc3d3a91 data_size 0x30(0x4560 bytes) 2024-09-14T07:00:17.815 INFO:teuthology.orchestra.run.smithi007.stdout:2923 write 0xb19e288 thru 0xb19ebee (0x967 bytes) 2024-09-14T07:00:17.819 INFO:teuthology.orchestra.run.smithi007.stdout:2924 writesame 0x1f16cb6 thru 0x1f1b1c4 data_size 0x3(0x450f bytes) 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 64.087318486000001, 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:00:17.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:00:17.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:00:17.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:00:17.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:17.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17618 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.584705548000002, 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016649290000000001, 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17618 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.830 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236332+0000", 2024-09-14T07:00:17.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237113+0000", 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078082900000000003 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237131+0000", 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7747000000000001e-05 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237141+0000", 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0137e-05 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237221+0000", 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0482000000000006e-05 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237383+0000", 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016176100000000001 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237737+0000", 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035380999999999998 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.241638+0000", 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039011739999999999 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252854+0000", 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011216097 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252919+0000", 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5129999999999995e-05 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252979+0000", 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9573e-05 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40284 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:9 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.616578486999998, 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016005640000000002, 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40284 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204457+0000", 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204459+0000", 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204466+0000", 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8730000000000001e-06 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204472+0000", 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4779999999999999e-06 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207358+0000", 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028862549999999999 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207430+0000", 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2268000000000007e-05 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220462+0000", 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013031951999999999 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40646 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.615436852999999, 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017879991000000001, 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40646 2024-09-14T07:00:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205605+0000", 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2260000000000002e-06 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205611+0000", 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3289999999999999e-06 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205622+0000", 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-14T07:00:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213726+0000", 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008104745 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213792+0000", 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5401999999999995e-05 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223478+0000", 2024-09-14T07:00:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096862100000000007 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40683 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.615364933999999, 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016494009, 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40683 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205672+0000", 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0379999999999998e-06 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205679+0000", 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205684+0000", 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9359999999999998e-06 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221845+0000", 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016161422000000002 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221913+0000", 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7669000000000006e-05 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222164+0000", 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025130600000000002 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40685 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.615344530000002, 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016657634000000001, 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40685 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205691+0000", 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205693+0000", 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205699+0000", 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205703+0000", 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5240000000000002e-06 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222190+0000", 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016486752 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222257+0000", 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7095999999999996e-05 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222348+0000", 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1061000000000001e-05 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40687 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.615325299999999, 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016826543999999999, 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40687 2024-09-14T07:00:17.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:17.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4999999999999998e-07 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205712+0000", 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205718+0000", 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205723+0000", 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222374+0000", 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016650933999999999 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222445+0000", 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1304999999999996e-05 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222536+0000", 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1104000000000001e-05 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40720 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.615208704, 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016893371000000001, 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40720 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205827+0000", 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.58e-07 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205834+0000", 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2729999999999997e-06 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205841+0000", 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205846+0000", 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1769999999999997e-06 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222562+0000", 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016716178000000002 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222633+0000", 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0461000000000002e-05 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222720+0000", 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6997999999999996e-05 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40721 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:17.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.615181002, 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044822000000001, 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40721 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:17.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205857+0000", 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-06 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205863+0000", 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205868+0000", 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222745+0000", 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016877293000000002 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222814+0000", 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8806000000000005e-05 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222899+0000", 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4913000000000005e-05 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40742 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.615160436, 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017201659000000001, 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40742 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205892+0000", 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7008e-05 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205899+0000", 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1489999999999997e-06 2024-09-14T07:00:17.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205904+0000", 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222923+0000", 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017018229999999999 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222991+0000", 2024-09-14T07:00:17.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8545000000000002e-05 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223076+0000", 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4938999999999994e-05 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40743 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.615121704, 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343654999999999, 2024-09-14T07:00:17.870 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40743 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205914+0000", 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-07 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205916+0000", 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0559999999999999e-06 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205922+0000", 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3659999999999997e-06 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205927+0000", 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8169999999999998e-06 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223101+0000", 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017174452 2024-09-14T07:00:17.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223168+0000", 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6756999999999998e-05 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223257+0000", 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8700000000000001e-05 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43363 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.367947938, 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015032403, 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43363 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.875 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:17.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453089+0000", 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9980000000000002e-06 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453096+0000", 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3060000000000003e-06 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453101+0000", 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8960000000000002e-06 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465082+0000", 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011981083 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465162+0000", 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0588999999999997e-05 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468119+0000", 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002957079 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43453 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 78848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.367660037, 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015012878, 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43453 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453377+0000", 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1050000000000002e-06 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453384+0000", 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453389+0000", 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-06 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467869+0000", 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014479845999999999 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467944+0000", 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5680999999999996e-05 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468388+0000", 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000443473 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43579 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:17.883 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.365173075000001, 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01522916, 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43579 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:17.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455864+0000", 2024-09-14T07:00:17.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455870+0000", 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455875+0000", 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9509999999999997e-06 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470879+0000", 2024-09-14T07:00:17.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015003729 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470960+0000", 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1228000000000002e-05 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471091+0000", 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131018 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43580 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 110592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.365153537000001, 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015447047, 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43580 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.888 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455882+0000", 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T07:00:17.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455884+0000", 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2010000000000002e-06 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455891+0000", 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5690000000000001e-06 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455896+0000", 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3639999999999998e-06 2024-09-14T07:00:17.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471123+0000", 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015226805 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471200+0000", 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7405999999999997e-05 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471328+0000", 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012835099999999999 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43581 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.365131636000001, 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015668949000000001, 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43581 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455904+0000", 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455910+0000", 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6619999999999999e-06 2024-09-14T07:00:17.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455917+0000", 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5769999999999999e-06 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455923+0000", 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6779999999999997e-06 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471360+0000", 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015436854999999999 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471443+0000", 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3815999999999998e-05 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471572+0000", 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012892900000000001 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117541 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.9294297010000001, 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015320307999999999, 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117541 2024-09-14T07:00:17.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891606+0000", 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891609+0000", 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.886e-06 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891615+0000", 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5010000000000001e-06 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891620+0000", 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901891+0000", 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010270833 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901942+0000", 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0612000000000003e-05 2024-09-14T07:00:17.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902142+0000", 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020015500000000001 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904269+0000", 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021265519999999999 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906874+0000", 2024-09-14T07:00:17.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026050800000000001 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906904+0000", 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199e-05 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906926+0000", 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1708e-05 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117542 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.9294080019999997, 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015409499, 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117542 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:17.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891630+0000", 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8569999999999999e-06 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891638+0000", 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5610000000000001e-06 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891643+0000", 2024-09-14T07:00:17.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4299999999999997e-06 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902805+0000", 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011162071000000001 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902879+0000", 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4152000000000004e-05 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:17.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903129+0000", 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024969499999999997 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904277+0000", 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001147906 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906968+0000", 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002690942 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906999+0000", 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1056000000000002e-05 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.907036+0000", 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7410000000000003e-05 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117543 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:17.908 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.9293755419999998, 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018002384999999999, 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117543 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:17.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891660+0000", 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8100000000000003e-07 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891663+0000", 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2739999999999999e-06 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891671+0000", 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.303e-06 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891678+0000", 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3709999999999996e-06 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903501+0000", 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011823382 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903577+0000", 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5947999999999999e-05 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903855+0000", 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027763899999999999 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904798+0000", 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094325599999999996 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909603+0000", 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048047539999999996 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909634+0000", 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1137999999999999e-05 2024-09-14T07:00:17.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909662+0000", 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7838999999999999e-05 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:120396 3.1c 3:3ebb3b40:::rbd_object_map.114f74d20483.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.262410826, 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015664009999999999, 2024-09-14T07:00:17.914 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 120396 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558625+0000", 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558631+0000", 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:17.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558647+0000", 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5756999999999998e-05 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558657+0000", 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.995e-06 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558724+0000", 2024-09-14T07:00:17.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7008000000000006e-05 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558815+0000", 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1077999999999998e-05 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.559173+0000", 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035768600000000002 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:17.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560127+0000", 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095468799999999998 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574168+0000", 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014040238999999999 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574233+0000", 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999994e-05 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:17.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574288+0000", 2024-09-14T07:00:17.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.558e-05 2024-09-14T07:00:17.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.920 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:17.920 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:17.920 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:17.921 INFO:teuthology.orchestra.run.smithi007.stdout:2926 write 0x3e1bbe6 thru 0x3e23b00 (0x7f1b bytes) 2024-09-14T07:00:17.921 INFO:teuthology.orchestra.run.smithi007.stdout:2927 read 0x89a4a4e thru 0x89a98c4 (0x4e77 bytes) 2024-09-14T07:00:17.921 INFO:teuthology.orchestra.run.smithi007.stdout:2928 trunc from 0xc3d3a92 to 0xd9c8373 2024-09-14T07:00:17.921 INFO:teuthology.orchestra.run.smithi007.stdout:2929 punch from 0x8843ed6 to 0x8851ff3, (0xe11d bytes) 2024-09-14T07:00:17.921 INFO:teuthology.orchestra.run.smithi007.stdout:2930 punch from 0x589d4be to 0x589f0bb, (0x1bfd bytes) 2024-09-14T07:00:17.921 INFO:teuthology.orchestra.run.smithi007.stdout:2931 read 0x6e8625f thru 0x6e87939 (0x16db bytes) 2024-09-14T07:00:17.921 INFO:teuthology.orchestra.run.smithi007.stdout:2932 read 0x802660c thru 0x8027e0e (0x1803 bytes) 2024-09-14T07:00:17.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:17.924 INFO:teuthology.orchestra.run.smithi007.stdout:2933 trunc from 0xd9c8373 to 0xc1cbbbf 2024-09-14T07:00:17.924 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:17.924 INFO:teuthology.orchestra.run.smithi007.stdout:2936 writesame 0x9c32cec thru 0x9c3d2dd data_size 0xb(0xa5f2 bytes) 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2937 read 0x165401b thru 0x16589e5 (0x49cb bytes) 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2938 trunc from 0xc1cbbbf to 0xc8ba0ab 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2940 write 0x8801169 thru 0x8805d17 (0x4baf bytes) 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2941 read 0x4d49144 thru 0x4d540b5 (0xaf72 bytes) 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2942 trunc from 0xc8ba0ab to 0x57e18a8 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2943 writesame 0x30a60ba thru 0x30aa960 data_size 0x1(0x48a7 bytes) 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2945 trunc from 0x57e18a8 to 0x33473aa 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2946 read 0x19b98f9 thru 0x19c9625 (0xfd2d bytes) 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2947 read 0x11615af thru 0x116849b (0x6eed bytes) 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2949 punch from 0x91c698 to 0x91fc97, (0x35ff bytes) 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2950 write 0x7203ccb thru 0x721310e (0xf444 bytes) 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2951 trunc from 0x721310f to 0x95c868c 2024-09-14T07:00:17.925 INFO:teuthology.orchestra.run.smithi007.stdout:2952 trunc from 0x95c868c to 0x8b30a05 2024-09-14T07:00:17.926 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:17.922+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:17.928 INFO:teuthology.orchestra.run.smithi007.stdout:2953 trunc from 0x8b30a05 to 0x8a4406c 2024-09-14T07:00:17.943 INFO:teuthology.orchestra.run.smithi007.stdout:2955 write 0xc3607d4 thru 0xc36ad9b (0xa5c8 bytes) 2024-09-14T07:00:17.947 INFO:teuthology.orchestra.run.smithi007.stdout:2956 write 0x370b4ae thru 0x3718c90 (0xd7e3 bytes) 2024-09-14T07:00:17.948 INFO:teuthology.orchestra.run.smithi007.stdout:2957 read 0x584fac8 thru 0x5851c6f (0x21a8 bytes) 2024-09-14T07:00:17.956 INFO:teuthology.orchestra.run.smithi007.stdout:2958 write 0xe2567b0 thru 0xe25e1f1 (0x7a42 bytes) 2024-09-14T07:00:17.959 INFO:teuthology.orchestra.run.smithi007.stdout:2960 read 0x6fea872 thru 0x6ff4ef8 (0xa687 bytes) 2024-09-14T07:00:17.965 INFO:teuthology.orchestra.run.smithi007.stdout:2961 trunc from 0xe25e1f2 to 0x9c39450 2024-09-14T07:00:17.973 INFO:teuthology.orchestra.run.smithi007.stdout:2962 writesame 0x8a8213b thru 0x8a8d277 data_size 0x11(0xb13d bytes) 2024-09-14T07:00:17.977 INFO:teuthology.orchestra.run.smithi007.stdout:2963 read 0x76f8cd3 thru 0x7700e43 (0x8171 bytes) 2024-09-14T07:00:17.977 INFO:teuthology.orchestra.run.smithi007.stdout:2964 read 0x43e4d94 thru 0x43eb513 (0x6780 bytes) 2024-09-14T07:00:17.977 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:17.977 INFO:teuthology.orchestra.run.smithi007.stdout:2966 read 0x192dad6 thru 0x1937d2e (0xa259 bytes) 2024-09-14T07:00:17.978 INFO:teuthology.orchestra.run.smithi007.stdout:2967 read 0x773cc76 thru 0x774a099 (0xd424 bytes) 2024-09-14T07:00:17.978 INFO:teuthology.orchestra.run.smithi007.stdout:2968 read 0x7c0674e thru 0x7c15085 (0xe938 bytes) 2024-09-14T07:00:17.978 INFO:teuthology.orchestra.run.smithi007.stdout:2969 read 0x2158156 thru 0x2164c88 (0xcb33 bytes) 2024-09-14T07:00:17.989 INFO:teuthology.orchestra.run.smithi007.stdout:2970 trunc from 0x9c39450 to 0x7e32b1 2024-09-14T07:00:18.025 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:18.024+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:18.052 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:18.053 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:18.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:18.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:18.075 INFO:teuthology.orchestra.run.smithi007.stdout:2971 punch from 0x4fad70 to 0x508f59, (0xe1e9 bytes) 2024-09-14T07:00:18.087 INFO:teuthology.orchestra.run.smithi007.stdout:2972 write 0x66144f1 thru 0x6617b62 (0x3672 bytes) 2024-09-14T07:00:18.101 INFO:teuthology.orchestra.run.smithi007.stdout:2974 writesame 0xab403af thru 0xab42a2d data_size 0x2d(0x267f bytes) 2024-09-14T07:00:18.101 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:18.101 INFO:teuthology.orchestra.run.smithi007.stdout:2977 write 0x21236a5 thru 0x212e65c (0xafb8 bytes) 2024-09-14T07:00:18.105 INFO:teuthology.orchestra.run.smithi007.stdout:2979 read 0x807f3e3 thru 0x808b01c (0xbc3a bytes) 2024-09-14T07:00:18.113 INFO:teuthology.orchestra.run.smithi007.stdout:2980 write 0xc9bba31 thru 0xc9bbfa5 (0x575 bytes) 2024-09-14T07:00:18.116 INFO:teuthology.orchestra.run.smithi007.stdout:2983 punch from 0x6b28f1c to 0x6b31dc5, (0x8ea9 bytes) 2024-09-14T07:00:18.116 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:18.116 INFO:teuthology.orchestra.run.smithi007.stdout:2986 writesame 0x47896d0 thru 0x47960dd data_size 0x4a(0xca0e bytes) 2024-09-14T07:00:18.125 INFO:teuthology.orchestra.run.smithi007.stdout:2987 writesame 0xe08b258 thru 0xe09560a data_size 0x3d(0xa3b3 bytes) 2024-09-14T07:00:18.125 INFO:teuthology.orchestra.run.smithi007.stdout:2988 writesame 0xdc89f41 thru 0xdc92249 data_size 0x1(0x8309 bytes) 2024-09-14T07:00:18.126 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:18.125+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:18.149 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:18.149 INFO:teuthology.orchestra.run.smithi007.stdout:2990 read 0x6b677b6 thru 0x6b6cd77 (0x55c2 bytes) 2024-09-14T07:00:18.149 INFO:teuthology.orchestra.run.smithi007.stdout:2992 read 0xa2199e2 thru 0xa22913c (0xf75b bytes) 2024-09-14T07:00:18.149 INFO:teuthology.orchestra.run.smithi007.stdout:2993 read 0x489b87f thru 0x48a65d6 (0xad58 bytes) 2024-09-14T07:00:18.150 INFO:teuthology.orchestra.run.smithi007.stdout:2994 writesame 0x2f0f096 thru 0x2f10f4c data_size 0x1(0x1eb7 bytes) 2024-09-14T07:00:18.169 INFO:teuthology.orchestra.run.smithi007.stdout:2995 trunc from 0xe09560b to 0x1b4d9a6 2024-09-14T07:00:18.186 INFO:teuthology.orchestra.run.smithi007.stdout:2996 trunc from 0x1b4d9a6 to 0x649313b 2024-09-14T07:00:18.193 INFO:teuthology.orchestra.run.smithi007.stdout:2997 punch from 0x57a9e01 to 0x57b8e96, (0xf095 bytes) 2024-09-14T07:00:18.197 INFO:teuthology.orchestra.run.smithi007.stdout:2999 trunc from 0x649313b to 0x255ce39 2024-09-14T07:00:18.203 INFO:teuthology.orchestra.run.smithi007.stdout:3000 read 0x1aa69aa thru 0x1aa988f (0x2ee6 bytes) 2024-09-14T07:00:18.203 INFO:teuthology.orchestra.run.smithi007.stdout:3003 read 0x56eb3f thru 0x5790ce (0xa590 bytes) 2024-09-14T07:00:18.204 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:18.207 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:18.207 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:18.212 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:18.213 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:18.213 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:18.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:18.225 INFO:teuthology.orchestra.run.smithi007.stdout:3006 writesame 0xc33d96b thru 0xc34b8b0 data_size 0x1(0xdf46 bytes) 2024-09-14T07:00:18.225 INFO:teuthology.orchestra.run.smithi007.stdout:3007 read 0x20c3469 thru 0x20c3e23 (0x9bb bytes) 2024-09-14T07:00:18.226 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:18.225+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:18.238 INFO:teuthology.orchestra.run.smithi007.stdout:3008 trunc from 0xc34b8b1 to 0x9c4104 2024-09-14T07:00:18.239 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:18.246 INFO:teuthology.orchestra.run.smithi007.stdout:3009 trunc from 0x9c4104 to 0x106b0ab 2024-09-14T07:00:18.252 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:18.252 INFO:teuthology.orchestra.run.smithi007.stdout:3012 punch from 0xc7e3a8 to 0xc83da1, (0x59f9 bytes) 2024-09-14T07:00:18.252 INFO:teuthology.orchestra.run.smithi007.stdout:3013 write 0x3dad8 thru 0x3f5e1 (0x1b0a bytes) 2024-09-14T07:00:18.259 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:18.259 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:18.262 INFO:teuthology.orchestra.run.smithi007.stdout:3017 write 0x26df2b1 thru 0x26e7e06 (0x8b56 bytes) 2024-09-14T07:00:18.265 INFO:teuthology.orchestra.run.smithi007.stdout:3020 punch from 0x1a3f64f to 0x1a41b41, (0x24f2 bytes) 2024-09-14T07:00:18.265 INFO:teuthology.orchestra.run.smithi007.stdout:3021 read 0x15f21c1 thru 0x1601a20 (0xf860 bytes) 2024-09-14T07:00:18.277 INFO:teuthology.orchestra.run.smithi007.stdout:3022 writesame 0x81efc57 thru 0x81f9961 data_size 0x9(0x9d0b bytes) 2024-09-14T07:00:18.278 INFO:teuthology.orchestra.run.smithi007.stdout:3023 punch from 0x538d1a9 to 0x539063f, (0x3496 bytes) 2024-09-14T07:00:18.279 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:18.279 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:18.279 INFO:teuthology.orchestra.run.smithi007.stdout:3025 trunc from 0x81f9962 to 0xa063934 2024-09-14T07:00:18.284 INFO:teuthology.orchestra.run.smithi007.stdout:3026 read 0x1ef4c22 thru 0x1f02e00 (0xe1df bytes) 2024-09-14T07:00:18.285 INFO:teuthology.orchestra.run.smithi007.stdout:3027 punch from 0x11496d to 0x120830, (0xbec3 bytes) 2024-09-14T07:00:18.285 INFO:teuthology.orchestra.run.smithi007.stdout:3028 write 0x5fbb5e8 thru 0x5fca772 (0xf18b bytes) 2024-09-14T07:00:18.288 INFO:teuthology.orchestra.run.smithi007.stdout:3029 writesame 0x53fa288 thru 0x54055ac data_size 0x1(0xb325 bytes) 2024-09-14T07:00:18.302 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:18.302 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:18.302 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:18.311 INFO:teuthology.orchestra.run.smithi007.stdout:3031 write 0xc925438 thru 0xc92ad0f (0x58d8 bytes) 2024-09-14T07:00:18.314 INFO:teuthology.orchestra.run.smithi007.stdout:3032 read 0x9ab60cc thru 0x9ab9d77 (0x3cac bytes) 2024-09-14T07:00:18.314 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:18.320 INFO:teuthology.orchestra.run.smithi007.stdout:3034 writesame 0xd086053 thru 0xd0905b0 data_size 0x2(0xa55e bytes) 2024-09-14T07:00:18.320 INFO:teuthology.orchestra.run.smithi007.stdout:3035 write 0x94b9eb7 thru 0x94c3ca1 (0x9deb bytes) 2024-09-14T07:00:18.322 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:18.326 INFO:teuthology.orchestra.run.smithi007.stdout:3036 trunc from 0xd0905b1 to 0xa7d0e3a 2024-09-14T07:00:18.327 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:18.326+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:3037 read 0x6c64c45 thru 0x6c67041 (0x23fd bytes) 2024-09-14T07:00:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:3038 clone 21 order 18 su 16384 sc 14 2024-09-14T07:00:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:18.359 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:18.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:18.458 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:18.457+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T07:00:18.533 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:18.533 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:18.533 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:18.532+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:18.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:18.630 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:18.629+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:18.689 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:18.689 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:18.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:18.742 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:18.741+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:18.805 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:18.806 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:18.814 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:18.855 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:18.854+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:18.931 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:18.931 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:18.932 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:18.930+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:18.940 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:19.033 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:19.032+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:19.085 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:19.086 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:19.094 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:19.134 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:19.133+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:19.235 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:19.234+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:19.239 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:19.239 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:19.248 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:19.335 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:19.335+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:19.351 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:19.351 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:19.360 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:19.437 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:19.436+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:19.463 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:19.463 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:19.471 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:19.508 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone20 from 0xa7d0e3a (overlap 0x7e32b1) to 0x64f55a 2024-09-14T07:00:19.556 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:19.556+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:19.596 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-14T07:00:19.610 INFO:teuthology.orchestra.run.smithi007.stdout:3039 read 0x5b0b5aa thru 0x5b0cc46 (0x169d bytes) 2024-09-14T07:00:19.619 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:19.619 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:19.620 INFO:teuthology.orchestra.run.smithi007.stdout:3040 trunc from 0xa7d0e3a to 0x23cab46 2024-09-14T07:00:19.628 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:19.638 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:19.638+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:19.662 INFO:teuthology.orchestra.run.smithi007.stdout:3041 writesame 0x5a585af thru 0x5a6072d data_size 0x1(0x817f bytes) 2024-09-14T07:00:19.668 INFO:teuthology.orchestra.run.smithi007.stdout:3042 write 0x5edc939 thru 0x5eeb229 (0xe8f1 bytes) 2024-09-14T07:00:19.680 INFO:teuthology.orchestra.run.smithi007.stdout:3043 trunc from 0x5eeb22a to 0xbbc393d 2024-09-14T07:00:19.688 INFO:teuthology.orchestra.run.smithi007.stdout:3045 trunc from 0xbbc393d to 0xa963050 2024-09-14T07:00:19.693 INFO:teuthology.orchestra.run.smithi007.stdout:3046 write 0x99b0589 thru 0x99b62b8 (0x5d30 bytes) 2024-09-14T07:00:19.705 INFO:teuthology.orchestra.run.smithi007.stdout:3050 write 0xc5eefe4 thru 0xc5f2fba (0x3fd7 bytes) 2024-09-14T07:00:19.717 INFO:teuthology.orchestra.run.smithi007.stdout:3051 write 0xe173788 thru 0xe17d59b (0x9e14 bytes) 2024-09-14T07:00:19.722 INFO:teuthology.orchestra.run.smithi007.stdout:3052 write 0xa8ac7a2 thru 0xa8ae29b (0x1afa bytes) 2024-09-14T07:00:19.729 INFO:teuthology.orchestra.run.smithi007.stdout:3053 trunc from 0xe17d59c to 0xa1d2bfe 2024-09-14T07:00:19.739 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:19.739+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:19.739 INFO:teuthology.orchestra.run.smithi007.stdout:3054 trunc from 0xa1d2bfe to 0xa2e1a1a 2024-09-14T07:00:19.753 INFO:teuthology.orchestra.run.smithi007.stdout:3055 write 0xd89348a thru 0xd898433 (0x4faa bytes) 2024-09-14T07:00:19.758 INFO:teuthology.orchestra.run.smithi007.stdout:3056 writesame 0x8611fc0 thru 0x8621857 data_size 0x56(0xf898 bytes) 2024-09-14T07:00:19.766 INFO:teuthology.orchestra.run.smithi007.stdout:3057 writesame 0x655aec7 thru 0x655e081 data_size 0x1(0x31bb bytes) 2024-09-14T07:00:19.778 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:19.778 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:19.787 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:19.787 INFO:teuthology.orchestra.run.smithi007.stdout:3058 trunc from 0xd898434 to 0x160670 2024-09-14T07:00:19.841 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:19.840+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:19.858 INFO:teuthology.orchestra.run.smithi007.stdout:3059 trunc from 0x160670 to 0x8d43bd8 2024-09-14T07:00:19.865 INFO:teuthology.orchestra.run.smithi007.stdout:3060 read 0x646900a thru 0x646ce44 (0x3e3b bytes) 2024-09-14T07:00:19.865 INFO:teuthology.orchestra.run.smithi007.stdout:3061 write 0x408668a thru 0x40882bd (0x1c34 bytes) 2024-09-14T07:00:19.878 INFO:teuthology.orchestra.run.smithi007.stdout:3062 trunc from 0x8d43bd8 to 0x9538c6 2024-09-14T07:00:19.898 INFO:teuthology.orchestra.run.smithi007.stdout:3063 write 0x2acabe1 thru 0x2acadeb (0x20b bytes) 2024-09-14T07:00:19.900 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:19.900 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:19.900 INFO:teuthology.orchestra.run.smithi007.stdout:3064 punch from 0x192fad7 to 0x193c018, (0xc541 bytes) 2024-09-14T07:00:19.909 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:19.916 INFO:teuthology.orchestra.run.smithi007.stdout:3065 writesame 0xb47254d thru 0xb473129 data_size 0x1(0xbdd bytes) 2024-09-14T07:00:19.916 INFO:teuthology.orchestra.run.smithi007.stdout:3066 read 0xaa23476 thru 0xaa29331 (0x5ebc bytes) 2024-09-14T07:00:19.917 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:19.917 INFO:teuthology.orchestra.run.smithi007.stdout:3068 read 0x2d852f6 thru 0x2d8fa8b (0xa796 bytes) 2024-09-14T07:00:19.917 INFO:teuthology.orchestra.run.smithi007.stdout:3070 read 0x363ebb9 thru 0x3642876 (0x3cbe bytes) 2024-09-14T07:00:19.917 INFO:teuthology.orchestra.run.smithi007.stdout:3071 write 0x699efd7 thru 0x69a8631 (0x965b bytes) 2024-09-14T07:00:19.922 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:19.927 INFO:teuthology.orchestra.run.smithi007.stdout:3074 trunc from 0xb47312a to 0xec7163d 2024-09-14T07:00:19.933 INFO:teuthology.orchestra.run.smithi007.stdout:3076 punch from 0x11b3518 to 0x11be543, (0xb02b bytes) 2024-09-14T07:00:19.933 INFO:teuthology.orchestra.run.smithi007.stdout:3077 read 0x90ac1dc thru 0x90af5a8 (0x33cd bytes) 2024-09-14T07:00:19.933 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:19.933 INFO:teuthology.orchestra.run.smithi007.stdout:3079 read 0x141d62f thru 0x142757a (0x9f4c bytes) 2024-09-14T07:00:19.933 INFO:teuthology.orchestra.run.smithi007.stdout:3080 writesame 0x8822354 thru 0x882f95a data_size 0x11(0xd607 bytes) 2024-09-14T07:00:19.939 INFO:teuthology.orchestra.run.smithi007.stdout:3082 write 0xd2bb225 thru 0xd2c4f85 (0x9d61 bytes) 2024-09-14T07:00:19.945 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:19.945 INFO:teuthology.orchestra.run.smithi007.stdout:3084 punch from 0x6cbc130 to 0x6cc019b, (0x406b bytes) 2024-09-14T07:00:19.950 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:19.949+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:19.956 INFO:teuthology.orchestra.run.smithi007.stdout:3085 trunc from 0xec7163d to 0x5101613 2024-09-14T07:00:19.967 INFO:teuthology.orchestra.run.smithi007.stdout:3086 write 0x29cfc3e thru 0x29dd7c0 (0xdb83 bytes) 2024-09-14T07:00:19.974 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:19.974 INFO:teuthology.orchestra.run.smithi007.stdout:3088 read 0x3cb8bf thru 0x3d4df6 (0x9538 bytes) 2024-09-14T07:00:19.974 INFO:teuthology.orchestra.run.smithi007.stdout:3089 writesame 0x2f473f4 thru 0x2f4b722 data_size 0x3(0x432f bytes) 2024-09-14T07:00:19.979 INFO:teuthology.orchestra.run.smithi007.stdout:3090 read 0x4089845 thru 0x408aa1e (0x11da bytes) 2024-09-14T07:00:19.979 INFO:teuthology.orchestra.run.smithi007.stdout:3092 read 0x41917c7 thru 0x41952b7 (0x3af1 bytes) 2024-09-14T07:00:19.979 INFO:teuthology.orchestra.run.smithi007.stdout:3094 read 0x1e4d403 thru 0x1e5ad1b (0xd919 bytes) 2024-09-14T07:00:19.993 INFO:teuthology.orchestra.run.smithi007.stdout:3096 write 0xca9a401 thru 0xcaa6ccc (0xc8cc bytes) 2024-09-14T07:00:20.004 INFO:teuthology.orchestra.run.smithi007.stdout:3099 trunc from 0xcaa6ccd to 0x8a8a6a5 2024-09-14T07:00:20.017 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.017 INFO:teuthology.orchestra.run.smithi007.stdout:3101 write 0x256538c thru 0x2565991 (0x606 bytes) 2024-09-14T07:00:20.019 INFO:teuthology.orchestra.run.smithi007.stdout:3102 trunc from 0x8a8a6a5 to 0x5e21ea5 2024-09-14T07:00:20.025 INFO:teuthology.orchestra.run.smithi007.stdout:3103 read 0x5339b19 thru 0x533a812 (0xcfa bytes) 2024-09-14T07:00:20.025 INFO:teuthology.orchestra.run.smithi007.stdout:3104 punch from 0x91c960 to 0x92c952, (0xfff2 bytes) 2024-09-14T07:00:20.025 INFO:teuthology.orchestra.run.smithi007.stdout:3107 read 0x4e3db1b thru 0x4e42279 (0x475f bytes) 2024-09-14T07:00:20.042 INFO:teuthology.orchestra.run.smithi007.stdout:3108 writesame 0xea01e1b thru 0xea0519c data_size 0x13(0x3382 bytes) 2024-09-14T07:00:20.042 INFO:teuthology.orchestra.run.smithi007.stdout:3109 read 0x186de2c thru 0x1875f87 (0x815c bytes) 2024-09-14T07:00:20.042 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.042 INFO:teuthology.orchestra.run.smithi007.stdout:3113 punch from 0x6090a5 to 0x6104b5, (0x7410 bytes) 2024-09-14T07:00:20.042 INFO:teuthology.orchestra.run.smithi007.stdout:3115 read 0xaaf75ac thru 0xab02b95 (0xb5ea bytes) 2024-09-14T07:00:20.042 INFO:teuthology.orchestra.run.smithi007.stdout:3116 punch from 0x1b3db2e to 0x1b455b2, (0x7a84 bytes) 2024-09-14T07:00:20.042 INFO:teuthology.orchestra.run.smithi007.stdout:3118 read 0x3ab8357 thru 0x3abfe5d (0x7b07 bytes) 2024-09-14T07:00:20.042 INFO:teuthology.orchestra.run.smithi007.stdout:3119 writesame 0xd4431a6 thru 0xd44e682 data_size 0x1(0xb4dd bytes) 2024-09-14T07:00:20.042 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:20.041+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:20.054 INFO:teuthology.orchestra.run.smithi007.stdout:3121 read 0x6603ba4 thru 0x6611c22 (0xe07f bytes) 2024-09-14T07:00:20.054 INFO:teuthology.orchestra.run.smithi007.stdout:3122 read 0x4ee5f3f thru 0x4ef1071 (0xb133 bytes) 2024-09-14T07:00:20.054 INFO:teuthology.orchestra.run.smithi007.stdout:3123 read 0x1793829 thru 0x179bcc5 (0x849d bytes) 2024-09-14T07:00:20.054 INFO:teuthology.orchestra.run.smithi007.stdout:3124 read 0xa714339 thru 0xa7173fa (0x30c2 bytes) 2024-09-14T07:00:20.057 INFO:teuthology.orchestra.run.smithi007.stdout:3125 trunc from 0xea0519d to 0xbcad973 2024-09-14T07:00:20.060 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:20.060 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:20.067 INFO:teuthology.orchestra.run.smithi007.stdout:3126 read 0x2d07c8c thru 0x2d09003 (0x1378 bytes) 2024-09-14T07:00:20.067 INFO:teuthology.orchestra.run.smithi007.stdout:3129 read 0x92f2a6c thru 0x92f8a76 (0x600b bytes) 2024-09-14T07:00:20.067 INFO:teuthology.orchestra.run.smithi007.stdout:3130 writesame 0x6bcc515 thru 0x6bd8b3c data_size 0x44(0xc628 bytes) 2024-09-14T07:00:20.068 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:20.073 INFO:teuthology.orchestra.run.smithi007.stdout:3131 write 0xc311d1 thru 0xc3dcca (0xcafa bytes) 2024-09-14T07:00:20.079 INFO:teuthology.orchestra.run.smithi007.stdout:3132 punch from 0xa937b1c to 0xa93e2da, (0x67be bytes) 2024-09-14T07:00:20.079 INFO:teuthology.orchestra.run.smithi007.stdout:3133 read 0x62a5a86 thru 0x62a7604 (0x1b7f bytes) 2024-09-14T07:00:20.079 INFO:teuthology.orchestra.run.smithi007.stdout:3135 write 0x5b1b32 thru 0x5b9b56 (0x8025 bytes) 2024-09-14T07:00:20.084 INFO:teuthology.orchestra.run.smithi007.stdout:3136 read 0x304d76d thru 0x30555f3 (0x7e87 bytes) 2024-09-14T07:00:20.084 INFO:teuthology.orchestra.run.smithi007.stdout:3138 writesame 0x5fc22cc thru 0x5fcd25a data_size 0x47(0xaf8f bytes) 2024-09-14T07:00:20.089 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.089 INFO:teuthology.orchestra.run.smithi007.stdout:3141 punch from 0xba410b7 to 0xba4fced, (0xec36 bytes) 2024-09-14T07:00:20.102 INFO:teuthology.orchestra.run.smithi007.stdout:3142 trunc from 0xbcad973 to 0x33c307 2024-09-14T07:00:20.141 INFO:teuthology.orchestra.run.smithi007.stdout:3143 writesame 0xec6435f thru 0xec6eb8b data_size 0x71(0xa82d bytes) 2024-09-14T07:00:20.142 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:20.141+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:20.154 INFO:teuthology.orchestra.run.smithi007.stdout:3144 trunc from 0xec6eb8c to 0x33eeb7a 2024-09-14T07:00:20.159 INFO:teuthology.orchestra.run.smithi007.stdout:3145 read 0x1b849e7 thru 0x1b89567 (0x4b81 bytes) 2024-09-14T07:00:20.168 INFO:teuthology.orchestra.run.smithi007.stdout:3146 write 0x6033740 thru 0x6043386 (0xfc47 bytes) 2024-09-14T07:00:20.175 INFO:teuthology.orchestra.run.smithi007.stdout:3149 write 0x41b725e thru 0x41bc400 (0x51a3 bytes) 2024-09-14T07:00:20.181 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:20.181 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:20.191 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:20.196 INFO:teuthology.orchestra.run.smithi007.stdout:3151 write 0xe7fe1e0 thru 0xe806ff6 (0x8e17 bytes) 2024-09-14T07:00:20.201 INFO:teuthology.orchestra.run.smithi007.stdout:3152 punch from 0xa03dbf7 to 0xa040d5a, (0x3163 bytes) 2024-09-14T07:00:20.201 INFO:teuthology.orchestra.run.smithi007.stdout:3153 write 0xd70a0c9 thru 0xd7179e9 (0xd921 bytes) 2024-09-14T07:00:20.207 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.207 INFO:teuthology.orchestra.run.smithi007.stdout:3156 punch from 0x9e34baa to 0x9e40900, (0xbd56 bytes) 2024-09-14T07:00:20.207 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.207 INFO:teuthology.orchestra.run.smithi007.stdout:3159 read 0x97cfb2a thru 0x97dcb9b (0xd072 bytes) 2024-09-14T07:00:20.214 INFO:teuthology.orchestra.run.smithi007.stdout:3160 trunc from 0xe806ff7 to 0x86ad707 2024-09-14T07:00:20.226 INFO:teuthology.orchestra.run.smithi007.stdout:3161 read 0x5bdf29b thru 0x5be6ef4 (0x7c5a bytes) 2024-09-14T07:00:20.226 INFO:teuthology.orchestra.run.smithi007.stdout:3162 trunc from 0x86ad707 to 0x8bbe059 2024-09-14T07:00:20.233 INFO:teuthology.orchestra.run.smithi007.stdout:3163 read 0x259c899 thru 0x259cb0f (0x277 bytes) 2024-09-14T07:00:20.241 INFO:teuthology.orchestra.run.smithi007.stdout:3164 write 0xafee56d thru 0xaff6701 (0x8195 bytes) 2024-09-14T07:00:20.246 INFO:teuthology.orchestra.run.smithi007.stdout:3165 read 0x9144ca3 thru 0x91460f2 (0x1450 bytes) 2024-09-14T07:00:20.246 INFO:teuthology.orchestra.run.smithi007.stdout:3166 read 0x2d046b6 thru 0x2d06eee (0x2839 bytes) 2024-09-14T07:00:20.247 INFO:teuthology.orchestra.run.smithi007.stdout:3168 read 0x3ffc1ad thru 0x4008020 (0xbe74 bytes) 2024-09-14T07:00:20.247 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.247 INFO:teuthology.orchestra.run.smithi007.stdout:3170 read 0x579b3c8 thru 0x579c5a0 (0x11d9 bytes) 2024-09-14T07:00:20.247 INFO:teuthology.orchestra.run.smithi007.stdout:3171 writesame 0x7fce550 thru 0x7fd6b5c data_size 0x1b(0x860d bytes) 2024-09-14T07:00:20.251 INFO:teuthology.orchestra.run.smithi007.stdout:3172 punch from 0xaf41e42 to 0xaf4469d, (0x285b bytes) 2024-09-14T07:00:20.251 INFO:teuthology.orchestra.run.smithi007.stdout:3174 punch from 0x5c7d7fb to 0x5c86f71, (0x9776 bytes) 2024-09-14T07:00:20.251 INFO:teuthology.orchestra.run.smithi007.stdout:3175 punch from 0x3eae15c to 0x3ebde2f, (0xfcd3 bytes) 2024-09-14T07:00:20.256 INFO:teuthology.orchestra.run.smithi007.stdout:3176 trunc from 0xaff6702 to 0x6eaa265 2024-09-14T07:00:20.268 INFO:teuthology.orchestra.run.smithi007.stdout:3179 trunc from 0x6eaa265 to 0x59d3ba0 2024-09-14T07:00:20.274 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:20.274+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:20.278 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.278 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.278 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.287 INFO:teuthology.orchestra.run.smithi007.stdout:3183 trunc from 0x59d3ba0 to 0xd82d3c7 2024-09-14T07:00:20.294 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.294 INFO:teuthology.orchestra.run.smithi007.stdout:3185 read 0xa92d434 thru 0xa93d336 (0xff03 bytes) 2024-09-14T07:00:20.294 INFO:teuthology.orchestra.run.smithi007.stdout:3187 write 0x548fc80 thru 0x549403f (0x43c0 bytes) 2024-09-14T07:00:20.306 INFO:teuthology.orchestra.run.smithi007.stdout:3188 writesame 0xeda7e54 thru 0xedb5d6b data_size 0x16(0xdf18 bytes) 2024-09-14T07:00:20.306 INFO:teuthology.orchestra.run.smithi007.stdout:3190 read 0x9b86e2 thru 0x9c626e (0xdb8d bytes) 2024-09-14T07:00:20.306 INFO:teuthology.orchestra.run.smithi007.stdout:3191 read 0x4dde778 thru 0x4de378c (0x5015 bytes) 2024-09-14T07:00:20.306 INFO:teuthology.orchestra.run.smithi007.stdout:3193 writesame 0xc023332 thru 0xc026f9d data_size 0x4(0x3c6c bytes) 2024-09-14T07:00:20.311 INFO:teuthology.orchestra.run.smithi007.stdout:3196 write 0xb7403c0 thru 0xb74fda1 (0xf9e2 bytes) 2024-09-14T07:00:20.316 INFO:teuthology.orchestra.run.smithi007.stdout:3198 read 0xd976b42 thru 0xd97bbcb (0x508a bytes) 2024-09-14T07:00:20.328 INFO:teuthology.orchestra.run.smithi007.stdout:3199 trunc from 0xedb5d6c to 0x4a413b1 2024-09-14T07:00:20.340 INFO:teuthology.orchestra.run.smithi007.stdout:3200 trunc from 0x4a413b1 to 0x402cedd 2024-09-14T07:00:20.340 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:20.340 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:20.344 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:20.344+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:20.347 INFO:teuthology.orchestra.run.smithi007.stdout:3201 read 0x3c72306 thru 0x3c81bd3 (0xf8ce bytes) 2024-09-14T07:00:20.347 INFO:teuthology.orchestra.run.smithi007.stdout:3202 writesame 0x1855490 thru 0x185e797 data_size 0x14(0x9308 bytes) 2024-09-14T07:00:20.350 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:20.352 INFO:teuthology.orchestra.run.smithi007.stdout:3203 read 0x8e3095 thru 0x8e8e03 (0x5d6f bytes) 2024-09-14T07:00:20.352 INFO:teuthology.orchestra.run.smithi007.stdout:3205 read 0x1450b52 thru 0x1455582 (0x4a31 bytes) 2024-09-14T07:00:20.352 INFO:teuthology.orchestra.run.smithi007.stdout:3206 read 0x145d2b5 thru 0x14622ee (0x503a bytes) 2024-09-14T07:00:20.352 INFO:teuthology.orchestra.run.smithi007.stdout:3207 trunc from 0x402cedd to 0x4a303a2 2024-09-14T07:00:20.360 INFO:teuthology.orchestra.run.smithi007.stdout:3209 punch from 0x42dd730 to 0x42e4fb3, (0x7883 bytes) 2024-09-14T07:00:20.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:20.377 INFO:teuthology.orchestra.run.smithi007.stdout:3210 write 0xe73b7d8 thru 0xe743148 (0x7971 bytes) 2024-09-14T07:00:20.382 INFO:teuthology.orchestra.run.smithi007.stdout:3211 write 0xb42d8b5 thru 0xb430ca5 (0x33f1 bytes) 2024-09-14T07:00:20.386 INFO:teuthology.orchestra.run.smithi007.stdout:3212 write 0x788d4a1 thru 0x7894116 (0x6c76 bytes) 2024-09-14T07:00:20.397 INFO:teuthology.orchestra.run.smithi007.stdout:3213 write 0xe7fc4c8 thru 0xe809d9d (0xd8d6 bytes) 2024-09-14T07:00:20.408 INFO:teuthology.orchestra.run.smithi007.stdout:3215 writesame 0xe80a8d3 thru 0xe818482 data_size 0x98(0xdbb0 bytes) 2024-09-14T07:00:20.421 INFO:teuthology.orchestra.run.smithi007.stdout:3216 trunc from 0xe818483 to 0x42af0b0 2024-09-14T07:00:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:3217 punch from 0x388773b to 0x3892545, (0xae0a bytes) 2024-09-14T07:00:20.434 INFO:teuthology.orchestra.run.smithi007.stdout:3218 writesame 0x34699d2 thru 0x3474e04 data_size 0x1(0xb433 bytes) 2024-09-14T07:00:20.445 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:20.445+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:20.459 INFO:teuthology.orchestra.run.smithi007.stdout:3219 write 0x74c1190 thru 0x74cd079 (0xbeea bytes) 2024-09-14T07:00:20.466 INFO:teuthology.orchestra.run.smithi007.stdout:3220 writesame 0x385c583 thru 0x386657d data_size 0x1(0x9ffb bytes) 2024-09-14T07:00:20.479 INFO:teuthology.orchestra.run.smithi007.stdout:3222 read 0x10d328e thru 0x10e1200 (0xdf73 bytes) 2024-09-14T07:00:20.482 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7] out_osds: [6, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:20.482 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:20.482 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-14T07:00:20.482 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:00:20.482 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:00:20.483 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-14T07:00:20.483 INFO:teuthology.orchestra.run.smithi007.stdout:3223 trunc from 0x74cd07a to 0x32ddc14 2024-09-14T07:00:20.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:00:20.497 INFO:teuthology.orchestra.run.smithi007.stdout:3224 trunc from 0x32ddc14 to 0x393e1ca 2024-09-14T07:00:20.503 INFO:teuthology.orchestra.run.smithi007.stdout:3225 read 0x147f441 thru 0x148697b (0x753b bytes) 2024-09-14T07:00:20.504 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:20.504 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:20.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:20.516 INFO:teuthology.orchestra.run.smithi007.stdout:3226 write 0x94ce563 thru 0x94d6320 (0x7dbe bytes) 2024-09-14T07:00:20.523 INFO:teuthology.orchestra.run.smithi007.stdout:3227 read 0x6e886a5 thru 0x6e8eaef (0x644b bytes) 2024-09-14T07:00:20.523 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.523 INFO:teuthology.orchestra.run.smithi007.stdout:3230 write 0x366727 thru 0x37172b (0xb005 bytes) 2024-09-14T07:00:20.531 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:20.531 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:20.532 INFO:teuthology.orchestra.run.smithi007.stdout:3232 writesame 0x99ff91b thru 0x9a08e61 data_size 0x1(0x9547 bytes) 2024-09-14T07:00:20.532 INFO:teuthology.orchestra.run.smithi007.stdout:3234 write 0xdb7674 thru 0xdbea45 (0x73d2 bytes) 2024-09-14T07:00:20.541 INFO:teuthology.orchestra.run.smithi007.stdout:3235 trunc from 0x9a08e62 to 0x6b5189e 2024-09-14T07:00:20.546 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:20.545+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:20.553 INFO:teuthology.orchestra.run.smithi007.stdout:3236 read 0x3a642a3 thru 0x3a661c6 (0x1f24 bytes) 2024-09-14T07:00:20.555 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:20.556 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:20.566 INFO:teuthology.orchestra.run.smithi007.stdout:3237 write 0xbb712ad thru 0xbb79367 (0x80bb bytes) 2024-09-14T07:00:20.573 INFO:teuthology.orchestra.run.smithi007.stdout:3238 read 0x240fc4d thru 0x2419be5 (0x9f99 bytes) 2024-09-14T07:00:20.573 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.573 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.580 INFO:teuthology.orchestra.run.smithi007.stdout:3241 write 0xcae1a1b thru 0xcaf1a17 (0xfffd bytes) 2024-09-14T07:00:20.588 INFO:teuthology.orchestra.run.smithi007.stdout:3242 read 0xaffc86 thru 0xb0cae0 (0xce5b bytes) 2024-09-14T07:00:20.588 INFO:teuthology.orchestra.run.smithi007.stdout:3243 writesame 0x3da3c92 thru 0x3db3066 data_size 0x1(0xf3d5 bytes) 2024-09-14T07:00:20.589 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:20.589 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:20.592 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:20.593 INFO:teuthology.orchestra.run.smithi189.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T07:00:20.593 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:20.611 INFO:teuthology.orchestra.run.smithi007.stdout:3244 read 0x2e73c46 thru 0x2e767c0 (0x2b7b bytes) 2024-09-14T07:00:20.611 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.619 INFO:teuthology.orchestra.run.smithi007.stdout:3246 write 0xe11fd82 thru 0xe12c22e (0xc4ad bytes) 2024-09-14T07:00:20.623 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:20.623 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:20.628 INFO:teuthology.orchestra.run.smithi007.stdout:3247 write 0xd059da4 thru 0xd065079 (0xb2d6 bytes) 2024-09-14T07:00:20.636 INFO:teuthology.orchestra.run.smithi007.stdout:3248 write 0x6815919 thru 0x68244da (0xebc2 bytes) 2024-09-14T07:00:20.646 INFO:teuthology.orchestra.run.smithi007.stdout:3249 write 0x8ec0325 thru 0x8ec6154 (0x5e30 bytes) 2024-09-14T07:00:20.646 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:20.646+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:20.649 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:20.649 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:20.652 INFO:teuthology.orchestra.run.smithi007.stdout:3250 read 0x6df6ecb thru 0x6df850e (0x1644 bytes) 2024-09-14T07:00:20.660 INFO:teuthology.orchestra.run.smithi007.stdout:3251 trunc from 0xe12c22f to 0x71c6190 2024-09-14T07:00:20.661 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:20.662 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:20.662 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:20.662 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:20.670 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:20.670 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:20.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:20.679 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:20.689 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:20.689 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:20.689 INFO:teuthology.orchestra.run.smithi007.stdout:3256 write 0x9dd000b thru 0x9ddd498 (0xd48e bytes) 2024-09-14T07:00:20.698 INFO:teuthology.orchestra.run.smithi007.stdout:3257 read 0x76cb96d thru 0x76d912a (0xd7be bytes) 2024-09-14T07:00:20.698 INFO:teuthology.orchestra.run.smithi007.stdout:3258 punch from 0x22b8881 to 0x22c3cd4, (0xb453 bytes) 2024-09-14T07:00:20.698 INFO:teuthology.orchestra.run.smithi007.stdout:3260 clone 22 order 20 su 262144 sc 15 2024-09-14T07:00:20.710 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:20.748 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:20.747+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:20.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:20.848 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:20.848+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.508393839, 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:00:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:00:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:00:20.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:00:20.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.919 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.508331998999999, 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:00:20.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:00:20.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.508298823000001, 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:00:20.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:00:20.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.508175951999998, 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:00:20.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.508141203000001, 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:00:20.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:00:20.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.508117275, 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:00:20.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.508095041000001, 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:00:20.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.508062073000001, 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:00:20.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:20.955 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.507711302000001, 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:00:20.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:00:20.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.507607118999999, 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:20.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:00:20.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.507566609000001, 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.966 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:20.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:00:20.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:00:20.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:00:20.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.507541489000001, 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:00:20.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.507387191999999, 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:20.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:00:20.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.981 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.981 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:20.981 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.507354196000001, 2024-09-14T07:00:20.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:00:20.981 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:00:20.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:00:20.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.507331291, 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:20.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:00:20.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.507281288000001, 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:00:20.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:00:20.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.3125492360000006, 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.996 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:00:20.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:20.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.3078427220000002, 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:21.004 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.3044590570000008, 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:21.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:00:21.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.3044393220000003, 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.013 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:21.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:21.015 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:21.013+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:21.115 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:21.114+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:21.122 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:21.122 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:21.122 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:21.122 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:21.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:21.216 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:21.215+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:21.278 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:21.279 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:21.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:21.331 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:21.330+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:21.396 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:00:21.396 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.660916465, 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:00:21.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:00:21.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17618 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.158303527000001, 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016649290000000001, 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:21.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17618 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:21.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236332+0000", 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237113+0000", 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078082900000000003 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237131+0000", 2024-09-14T07:00:21.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7747000000000001e-05 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237141+0000", 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0137e-05 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237221+0000", 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0482000000000006e-05 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237383+0000", 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016176100000000001 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237737+0000", 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035380999999999998 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.241638+0000", 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039011739999999999 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252854+0000", 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011216097 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252919+0000", 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5129999999999995e-05 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252979+0000", 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9573e-05 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40284 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:9 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.190176465999997, 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016005640000000002, 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40284 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204457+0000", 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204459+0000", 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204466+0000", 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8730000000000001e-06 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204472+0000", 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4779999999999999e-06 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207358+0000", 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028862549999999999 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207430+0000", 2024-09-14T07:00:21.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2268000000000007e-05 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220462+0000", 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013031951999999999 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40646 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.189034831999997, 2024-09-14T07:00:21.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017879991000000001, 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40646 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205605+0000", 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2260000000000002e-06 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205611+0000", 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3289999999999999e-06 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205622+0000", 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213726+0000", 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008104745 2024-09-14T07:00:21.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213792+0000", 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5401999999999995e-05 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223478+0000", 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096862100000000007 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40683 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.188962912999997, 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016494009, 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40683 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205672+0000", 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0379999999999998e-06 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205679+0000", 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205684+0000", 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9359999999999998e-06 2024-09-14T07:00:21.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221845+0000", 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016161422000000002 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221913+0000", 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7669000000000006e-05 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222164+0000", 2024-09-14T07:00:21.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025130600000000002 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40685 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.188942509, 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016657634000000001, 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40685 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205691+0000", 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205693+0000", 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-14T07:00:21.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205699+0000", 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205703+0000", 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5240000000000002e-06 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222190+0000", 2024-09-14T07:00:21.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016486752 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222257+0000", 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7095999999999996e-05 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222348+0000", 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1061000000000001e-05 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40687 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.188923279000001, 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016826543999999999, 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40687 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4999999999999998e-07 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205712+0000", 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205718+0000", 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205723+0000", 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:21.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222374+0000", 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016650933999999999 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222445+0000", 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1304999999999996e-05 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222536+0000", 2024-09-14T07:00:21.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1104000000000001e-05 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40720 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.188806683000003, 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016893371000000001, 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40720 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:21.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205827+0000", 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.58e-07 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205834+0000", 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2729999999999997e-06 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205841+0000", 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205846+0000", 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1769999999999997e-06 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222562+0000", 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016716178000000002 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222633+0000", 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0461000000000002e-05 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222720+0000", 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6997999999999996e-05 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40721 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:21.435 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.188778980999999, 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044822000000001, 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40721 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:21.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205857+0000", 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-06 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205863+0000", 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205868+0000", 2024-09-14T07:00:21.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222745+0000", 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016877293000000002 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222814+0000", 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8806000000000005e-05 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222899+0000", 2024-09-14T07:00:21.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4913000000000005e-05 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40742 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.188758415000002, 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017201659000000001, 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40742 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205892+0000", 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7008e-05 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205899+0000", 2024-09-14T07:00:21.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1489999999999997e-06 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205904+0000", 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222923+0000", 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017018229999999999 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222991+0000", 2024-09-14T07:00:21.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8545000000000002e-05 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223076+0000", 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4938999999999994e-05 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40743 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.188719683000002, 2024-09-14T07:00:21.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343654999999999, 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40743 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205914+0000", 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-07 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205916+0000", 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0559999999999999e-06 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205922+0000", 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3659999999999997e-06 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205927+0000", 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8169999999999998e-06 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223101+0000", 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017174452 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223168+0000", 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6756999999999998e-05 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223257+0000", 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8700000000000001e-05 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43363 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 63488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.941545916999999, 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015032403, 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43363 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.449 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453087+0000", 2024-09-14T07:00:21.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000002e-07 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453089+0000", 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9980000000000002e-06 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453096+0000", 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3060000000000003e-06 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453101+0000", 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8960000000000002e-06 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465082+0000", 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011981083 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.465162+0000", 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0588999999999997e-05 2024-09-14T07:00:21.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468119+0000", 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002957079 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43453 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 78848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.941258015999999, 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015012878, 2024-09-14T07:00:21.453 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43453 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453375+0000", 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0999999999999999e-07 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453377+0000", 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1050000000000002e-06 2024-09-14T07:00:21.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453384+0000", 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.453389+0000", 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-06 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467869+0000", 2024-09-14T07:00:21.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014479845999999999 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.467944+0000", 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5680999999999996e-05 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.468388+0000", 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000443473 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43579 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.938771054, 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01522916, 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43579 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:21.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455864+0000", 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455870+0000", 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455875+0000", 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9509999999999997e-06 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470879+0000", 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015003729 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470960+0000", 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1228000000000002e-05 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471091+0000", 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131018 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43580 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 110592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.938751516, 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015447047, 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43580 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455882+0000", 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455884+0000", 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2010000000000002e-06 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455891+0000", 2024-09-14T07:00:21.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5690000000000001e-06 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455896+0000", 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3639999999999998e-06 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471123+0000", 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015226805 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471200+0000", 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7405999999999997e-05 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471328+0000", 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012835099999999999 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43581 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:21.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.938729615, 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015668949000000001, 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43581 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455904+0000", 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455910+0000", 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6619999999999999e-06 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455917+0000", 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5769999999999999e-06 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455923+0000", 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6779999999999997e-06 2024-09-14T07:00:21.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471360+0000", 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015436854999999999 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471443+0000", 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3815999999999998e-05 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471572+0000", 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012892900000000001 2024-09-14T07:00:21.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117541 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.5030276800000006, 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015320307999999999, 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117541 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891606+0000", 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891609+0000", 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.886e-06 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891615+0000", 2024-09-14T07:00:21.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5010000000000001e-06 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891620+0000", 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901891+0000", 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010270833 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901942+0000", 2024-09-14T07:00:21.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0612000000000003e-05 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902142+0000", 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020015500000000001 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904269+0000", 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021265519999999999 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:21.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906874+0000", 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026050800000000001 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906904+0000", 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199e-05 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906926+0000", 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1708e-05 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117542 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.5030059809999994, 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015409499, 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117542 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:21.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891630+0000", 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8569999999999999e-06 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891638+0000", 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5610000000000001e-06 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891643+0000", 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4299999999999997e-06 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902805+0000", 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011162071000000001 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902879+0000", 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4152000000000004e-05 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903129+0000", 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024969499999999997 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904277+0000", 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001147906 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906968+0000", 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002690942 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906999+0000", 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1056000000000002e-05 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.907036+0000", 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7410000000000003e-05 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117543 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.5029735209999995, 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018002384999999999, 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117543 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891660+0000", 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8100000000000003e-07 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891663+0000", 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2739999999999999e-06 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891671+0000", 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.303e-06 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891678+0000", 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3709999999999996e-06 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903501+0000", 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011823382 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903577+0000", 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5947999999999999e-05 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903855+0000", 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027763899999999999 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904798+0000", 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094325599999999996 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909603+0000", 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048047539999999996 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909634+0000", 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1137999999999999e-05 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909662+0000", 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7838999999999999e-05 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:120396 3.1c 3:3ebb3b40:::rbd_object_map.114f74d20483.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.8360088049999996, 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015664009999999999, 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 120396 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558625+0000", 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558631+0000", 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558647+0000", 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5756999999999998e-05 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558657+0000", 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.995e-06 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558724+0000", 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7008000000000006e-05 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558815+0000", 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1077999999999998e-05 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.559173+0000", 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035768600000000002 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560127+0000", 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095468799999999998 2024-09-14T07:00:21.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574168+0000", 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014040238999999999 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574233+0000", 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999994e-05 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574288+0000", 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.558e-05 2024-09-14T07:00:21.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.494 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:21.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:21.496 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T07:00:21.496 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:21.494+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:21.566 INFO:teuthology.orchestra.run.smithi007.stdout:flattening image image_client.0-clone21 2024-09-14T07:00:21.596 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:21.595+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:21.671 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:21.671 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:21.671 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:21.671 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:21.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:21.697 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:21.696+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:21.701 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-14T07:00:21.718 INFO:teuthology.orchestra.run.smithi007.stdout:3261 write 0x51d0f33 thru 0x51e068f (0xf75d bytes) 2024-09-14T07:00:21.733 INFO:teuthology.orchestra.run.smithi007.stdout:3262 write 0xc2dd6b8 thru 0xc2e0868 (0x31b1 bytes) 2024-09-14T07:00:21.737 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:21.737 INFO:teuthology.orchestra.run.smithi007.stdout:3264 write 0xab40f5f thru 0xab49f4b (0x8fed bytes) 2024-09-14T07:00:21.739 INFO:teuthology.orchestra.run.smithi007.stdout:3265 writesame 0x56488cc thru 0x56515c7 data_size 0xe(0x8cfc bytes) 2024-09-14T07:00:21.743 INFO:teuthology.orchestra.run.smithi007.stdout:3266 read 0x65518d thru 0x656058 (0xecc bytes) 2024-09-14T07:00:21.744 INFO:teuthology.orchestra.run.smithi007.stdout:3267 writesame 0xadf7102 thru 0xadf8030 data_size 0xd(0xf2f bytes) 2024-09-14T07:00:21.747 INFO:teuthology.orchestra.run.smithi007.stdout:3268 write 0xc1c8b30 thru 0xc1cd3de (0x48af bytes) 2024-09-14T07:00:21.750 INFO:teuthology.orchestra.run.smithi007.stdout:3269 writesame 0x1c3c462 thru 0x1c42651 data_size 0x1(0x61f0 bytes) 2024-09-14T07:00:21.761 INFO:teuthology.orchestra.run.smithi007.stdout:3270 read 0x25eed2b thru 0x25fc9e7 (0xdcbd bytes) 2024-09-14T07:00:21.761 INFO:teuthology.orchestra.run.smithi007.stdout:3271 writesame 0xb5db3 thru 0xc0a1c data_size 0x1d(0xac6a bytes) 2024-09-14T07:00:21.767 INFO:teuthology.orchestra.run.smithi007.stdout:3272 read 0xaf5d23f thru 0xaf6ae6c (0xdc2e bytes) 2024-09-14T07:00:21.767 INFO:teuthology.orchestra.run.smithi007.stdout:3273 read 0x8362897 thru 0x8372468 (0xfbd2 bytes) 2024-09-14T07:00:21.767 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:21.767 INFO:teuthology.orchestra.run.smithi007.stdout:3275 punch from 0x21a1536 to 0x21a9866, (0x8330 bytes) 2024-09-14T07:00:21.768 INFO:teuthology.orchestra.run.smithi007.stdout:3276 write 0x70f5efd thru 0x70fb8a7 (0x59ab bytes) 2024-09-14T07:00:21.771 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:21.779 INFO:teuthology.orchestra.run.smithi007.stdout:3278 trunc from 0xc2e0869 to 0x5b6a44b 2024-09-14T07:00:21.798 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:21.797+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:21.798 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:21.804 INFO:teuthology.orchestra.run.smithi007.stdout:3281 write 0x6b429de thru 0x6b4cff9 (0xa61c bytes) 2024-09-14T07:00:21.813 INFO:teuthology.orchestra.run.smithi007.stdout:3282 trunc from 0x6b4cffa to 0xbd6d78e 2024-09-14T07:00:21.819 INFO:teuthology.orchestra.run.smithi007.stdout:3285 write 0x34dd4a1 thru 0x34ea7f3 (0xd353 bytes) 2024-09-14T07:00:21.822 INFO:teuthology.orchestra.run.smithi007.stdout:3286 punch from 0xac6348f to 0xac68453, (0x4fc4 bytes) 2024-09-14T07:00:21.823 INFO:teuthology.orchestra.run.smithi007.stdout:3288 write 0xa0de3e7 thru 0xa0ea8b3 (0xc4cd bytes) 2024-09-14T07:00:21.826 INFO:teuthology.orchestra.run.smithi007.stdout:3289 read 0x2d41311 thru 0x2d4d088 (0xbd78 bytes) 2024-09-14T07:00:21.826 INFO:teuthology.orchestra.run.smithi007.stdout:3291 punch from 0x9a4dd2a to 0x9a56f0e, (0x91e4 bytes) 2024-09-14T07:00:21.827 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:21.827 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:21.827 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:21.833 INFO:teuthology.orchestra.run.smithi007.stdout:3293 write 0xd10e04e thru 0xd10e5d5 (0x588 bytes) 2024-09-14T07:00:21.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:21.837 INFO:teuthology.orchestra.run.smithi007.stdout:3295 read 0x6f88451 thru 0x6f8c0da (0x3c8a bytes) 2024-09-14T07:00:21.838 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:21.838 INFO:teuthology.orchestra.run.smithi007.stdout:3298 trunc from 0xd10e5d6 to 0xede147a 2024-09-14T07:00:21.845 INFO:teuthology.orchestra.run.smithi007.stdout:3299 write 0x38c8e79 thru 0x38cffae (0x7136 bytes) 2024-09-14T07:00:21.849 INFO:teuthology.orchestra.run.smithi007.stdout:3300 read 0x3922d94 thru 0x392cb14 (0x9d81 bytes) 2024-09-14T07:00:21.849 INFO:teuthology.orchestra.run.smithi007.stdout:3302 read 0x6ccbb5e thru 0x6cd72c3 (0xb766 bytes) 2024-09-14T07:00:21.849 INFO:teuthology.orchestra.run.smithi007.stdout:3303 read 0xd61a55d thru 0xd61f907 (0x53ab bytes) 2024-09-14T07:00:21.849 INFO:teuthology.orchestra.run.smithi007.stdout:3305 read 0xde97db8 thru 0xde97e9c (0xe5 bytes) 2024-09-14T07:00:21.849 INFO:teuthology.orchestra.run.smithi007.stdout:3306 punch from 0xac31417 to 0xac34d6b, (0x3954 bytes) 2024-09-14T07:00:21.849 INFO:teuthology.orchestra.run.smithi007.stdout:3307 write 0x2e1f780 thru 0x2e20617 (0xe98 bytes) 2024-09-14T07:00:21.853 INFO:teuthology.orchestra.run.smithi007.stdout:3310 trunc from 0xede147a to 0xe105ef4 2024-09-14T07:00:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:3311 read 0x610b471 thru 0x61151ac (0x9d3c bytes) 2024-09-14T07:00:21.859 INFO:teuthology.orchestra.run.smithi007.stdout:3312 write 0x90f183c thru 0x90ffa4c (0xe211 bytes) 2024-09-14T07:00:21.875 INFO:teuthology.orchestra.run.smithi007.stdout:3315 trunc from 0xe105ef4 to 0x34bfb20 2024-09-14T07:00:21.898 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:21.898+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:21.903 INFO:teuthology.orchestra.run.smithi007.stdout:3317 punch from 0x261edeb to 0x26211a5, (0x23ba bytes) 2024-09-14T07:00:21.904 INFO:teuthology.orchestra.run.smithi007.stdout:3318 read 0x235b791 thru 0x2364146 (0x89b6 bytes) 2024-09-14T07:00:21.918 INFO:teuthology.orchestra.run.smithi007.stdout:3319 write 0xb4d8528 thru 0xb4d9583 (0x105c bytes) 2024-09-14T07:00:21.921 INFO:teuthology.orchestra.run.smithi007.stdout:3320 read 0x1a353b7 thru 0x1a368fe (0x1548 bytes) 2024-09-14T07:00:21.932 INFO:teuthology.orchestra.run.smithi007.stdout:3321 trunc from 0xb4d9584 to 0x1dba857 2024-09-14T07:00:21.950 INFO:teuthology.orchestra.run.smithi007.stdout:3322 read 0x812e97 thru 0x81b36c (0x84d6 bytes) 2024-09-14T07:00:21.951 INFO:teuthology.orchestra.run.smithi007.stdout:3323 read 0xb37f55 thru 0xb46ae9 (0xeb95 bytes) 2024-09-14T07:00:21.951 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:21.963 INFO:teuthology.orchestra.run.smithi007.stdout:3325 trunc from 0x1dba857 to 0xd2a57f5 2024-09-14T07:00:21.968 INFO:teuthology.orchestra.run.smithi007.stdout:3327 writesame 0x6f34e80 thru 0x6f43fc1 data_size 0x1(0xf142 bytes) 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119746 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1298432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4244340009999998, 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067798875999999994, 2024-09-14T07:00:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119746 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545019+0000", 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545020+0000", 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.782e-06 2024-09-14T07:00:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545026+0000", 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9390000000000002e-06 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545031+0000", 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5770000000000004e-06 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612601+0000", 2024-09-14T07:00:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067570682000000007 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612683+0000", 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1378999999999995e-05 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612817+0000", 2024-09-14T07:00:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013415000000000001 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119747 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.424415582, 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068036012000000007, 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119747 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546792+0000", 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017554960000000001 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546797+0000", 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1950000000000002e-06 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546808+0000", 2024-09-14T07:00:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0675e-05 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546815+0000", 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612854+0000", 2024-09-14T07:00:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066039158000000001 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612930+0000", 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.613073+0000", 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000142798 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119753 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1398784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4224914200000001, 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067577770999999995, 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119753 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546993+0000", 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1860999999999999e-05 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547007+0000", 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4346e-05 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547016+0000", 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4709999999999996e-06 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547023+0000", 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4010000000000002e-06 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614322+0000", 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067299231000000001 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614391+0000", 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8826999999999997e-05 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614538+0000", 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000147634 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119754 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1413120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4224008819999998, 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067735654000000006, 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119754 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549973+0000", 2024-09-14T07:00:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029215999999999999 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549978+0000", 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3510000000000001e-06 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549988+0000", 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0037000000000001e-05 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549995+0000", 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9709999999999998e-06 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614569+0000", 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064573368000000006 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614650+0000", 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1909000000000005e-05 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614787+0000", 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013641799999999999 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119764 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1556480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4192289749999993, 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067672107999999995, 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119764 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550224+0000", 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550226+0000", 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1720000000000001e-06 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550233+0000", 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9879999999999998e-06 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550238+0000", 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1819999999999997e-06 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617715+0000", 2024-09-14T07:00:21.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067476628999999996 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617807+0000", 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2899999999999995e-05 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617895+0000", 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7837999999999995e-05 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119765 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4192065429999996, 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067829500000000001, 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119765 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.995 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7e-07 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550248+0000", 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1280000000000002e-06 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550256+0000", 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550261+0000", 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617926+0000", 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067665280999999994 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617995+0000", 2024-09-14T07:00:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9478000000000001e-05 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618075+0000", 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9666000000000006e-05 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119950 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 529408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:21.999 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4191844370000002, 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067992002999999995, 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119950 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550852+0000", 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058435400000000004 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550857+0000", 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6979999999999997e-06 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550867+0000", 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0378e-05 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550874+0000", 2024-09-14T07:00:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8480000000000003e-06 2024-09-14T07:00:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618106+0000", 2024-09-14T07:00:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067231714999999997 2024-09-14T07:00:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618178+0000", 2024-09-14T07:00:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2572999999999998e-05 2024-09-14T07:00:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618260+0000", 2024-09-14T07:00:22.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1437e-05 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119970 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 816128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4125292809999994, 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067745828999999994, 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119970 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556924+0000", 2024-09-14T07:00:22.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0410000000000001e-06 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556927+0000", 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9679999999999998e-06 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556930+0000", 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624487+0000", 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067557392999999993 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624551+0000", 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3861000000000004e-05 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624669+0000", 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117921 2024-09-14T07:00:22.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119971 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4125193419999995, 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067869065000000006, 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119971 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556934+0000", 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.096e-06 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556937+0000", 2024-09-14T07:00:22.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.796e-06 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556939+0000", 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624708+0000", 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067768992 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624747+0000", 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8148999999999997e-05 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624802+0000", 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.533e-05 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.014 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119972 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 844800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4125096710000005, 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068010762000000002, 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119972 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556943+0000", 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:22.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556944+0000", 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556947+0000", 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9720000000000001e-06 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556949+0000", 2024-09-14T07:00:22.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4899999999999999e-06 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624838+0000", 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067888899000000003 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624895+0000", 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7395000000000002e-05 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624953+0000", 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7745999999999999e-05 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119973 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 859136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.412499789, 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068210898000000006, 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119973 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556953+0000", 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556954+0000", 2024-09-14T07:00:22.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189999999999999e-06 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556957+0000", 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9490000000000001e-06 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556959+0000", 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3870000000000002e-06 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624983+0000", 2024-09-14T07:00:22.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068023726000000007 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625049+0000", 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6385999999999998e-05 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625163+0000", 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114221 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119974 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4124898679999998, 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068338594000000003, 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119974 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556963+0000", 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556964+0000", 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556969+0000", 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.04e-06 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556974+0000", 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.639e-06 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625194+0000", 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068220742000000001 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625246+0000", 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1844e-05 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625301+0000", 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4551e-05 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119975 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4124716280000005, 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068554198999999996, 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119975 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556983+0000", 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2560000000000001e-06 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556989+0000", 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2210000000000002e-06 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556994+0000", 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7779999999999998e-06 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625330+0000", 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068335911999999999 2024-09-14T07:00:22.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625396+0000", 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6051999999999997e-05 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625535+0000", 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013860200000000001 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119976 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4124506330000006, 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068656164000000006, 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119976 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557004+0000", 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557010+0000", 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1770000000000003e-06 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557016+0000", 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2650000000000004e-06 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625566+0000", 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068550323999999996 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625604+0000", 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8398000000000001e-05 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625658+0000", 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3239000000000003e-05 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.037 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119977 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4123702139999992, 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068713522999999999, 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119977 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.038 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:22.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557086+0000", 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3569999999999998e-06 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557094+0000", 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6829999999999997e-06 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557101+0000", 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625689+0000", 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068587882000000003 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625743+0000", 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4082000000000002e-05 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625795+0000", 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2657000000000001e-05 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119978 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 930816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4123416219999996, 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068826410000000005, 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119978 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557113+0000", 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-14T07:00:22.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557120+0000", 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1079999999999998e-06 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557125+0000", 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625823+0000", 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068698928000000006 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625884+0000", 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0056999999999998e-05 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625937+0000", 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3383e-05 2024-09-14T07:00:22.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119979 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 945152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4123204749999996, 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068940788000000003, 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119979 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5199999999999998e-07 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557134+0000", 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.018e-06 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557140+0000", 2024-09-14T07:00:22.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557145+0000", 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7790000000000003e-06 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625966+0000", 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068820972999999994 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626019+0000", 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3828e-05 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626072+0000", 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3078000000000002e-05 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119980 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.051 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4123001170000009, 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069053630000000005, 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119980 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.052 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:22.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557159+0000", 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557167+0000", 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999996e-06 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557172+0000", 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-14T07:00:22.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626101+0000", 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068928552000000004 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626154+0000", 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3338000000000003e-05 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626206+0000", 2024-09-14T07:00:22.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1657999999999999e-05 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119981 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4122714670000001, 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069157629999999998, 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119981 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:22.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557183+0000", 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557190+0000", 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2249999999999997e-06 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557195+0000", 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3120000000000001e-06 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626234+0000", 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069038926 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626286+0000", 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2604e-05 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626338+0000", 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1947000000000003e-05 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119982 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 988160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:22.060 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.4122503139999996, 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069364004000000007, 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119982 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560989+0000", 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037867360000000002 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560994+0000", 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561004+0000", 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3680000000000008e-06 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561011+0000", 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1640000000000004e-06 2024-09-14T07:00:22.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626366+0000", 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.065355286999999998 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626433+0000", 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6554e-05 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626566+0000", 2024-09-14T07:00:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132996 2024-09-14T07:00:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.065 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:22.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:22.068 INFO:teuthology.orchestra.run.smithi007.stdout:3330 punch from 0x30e16e to 0x3146d1, (0x6563 bytes) 2024-09-14T07:00:22.068 INFO:teuthology.orchestra.run.smithi007.stdout:3333 write 0xcbba035 thru 0xcbbcd68 (0x2d34 bytes) 2024-09-14T07:00:22.068 INFO:teuthology.orchestra.run.smithi007.stdout:3334 writesame 0x568e5d thru 0x57353e data_size 0x29(0xa6e2 bytes) 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:3336 write 0xbc17430 thru 0xbc18ef3 (0x1ac4 bytes) 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:3337 write 0x7cda5cf thru 0x7cdf664 (0x5096 bytes) 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:3338 write 0x8718737 thru 0x8722d08 (0xa5d2 bytes) 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:3339 write 0x7472d67 thru 0x7480352 (0xd5ec bytes) 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:3343 write 0x7762b90 thru 0x776ce85 (0xa2f6 bytes) 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:3344 trunc from 0xd2a57f5 to 0x5588d9 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:3345 punch from 0x18a707 to 0x1911c4, (0x6abd bytes) 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:3346 read 0x1c64bb thru 0x1d17d5 (0xb31b bytes) 2024-09-14T07:00:22.069 INFO:teuthology.orchestra.run.smithi007.stdout:3347 read 0x1cb3d thru 0x23a4a (0x6f0e bytes) 2024-09-14T07:00:22.070 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:22.072 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:22.065+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:22.076 INFO:teuthology.orchestra.run.smithi007.stdout:3349 write 0x42b1d2f thru 0x42b86b0 (0x6982 bytes) 2024-09-14T07:00:22.079 INFO:teuthology.orchestra.run.smithi007.stdout:3350 writesame 0x31c2888 thru 0x31c310a data_size 0x1(0x883 bytes) 2024-09-14T07:00:22.082 INFO:teuthology.orchestra.run.smithi007.stdout:3351 clone 23 order 20 su 524288 sc 2 2024-09-14T07:00:22.168 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:22.167+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:22.183 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:22.183 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:22.183 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:22.183 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:22.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:22.268 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:22.268+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:22.339 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:22.339 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:22.339 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:22.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:22.391 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:22.390+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.701537158999997, 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:00:22.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:00:22.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:00:22.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:00:22.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.103124567000002, 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:22.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:22.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:00:22.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.097464924000001, 2024-09-14T07:00:22.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:00:22.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:00:22.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:22.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:00:22.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:00:22.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:00:22.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:00:22.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:00:22.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.791628645999999, 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:00:22.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:00:22.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:22.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.791597854000003, 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:22.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.791574484000002, 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:00:22.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:00:22.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.790767975000001, 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:00:22.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:00:22.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:00:22.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.790745582, 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:00:22.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:00:22.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.790725442000003, 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:00:22.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.505 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.790575287000003, 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:22.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:00:22.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:00:22.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:00:22.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.790519732999996, 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:00:22.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:00:22.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.790486692000002, 2024-09-14T07:00:22.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:00:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:00:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:00:22.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.788133606999999, 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:00:22.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:00:22.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:22.523 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.788111041999997, 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:00:22.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45774 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 933888~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.787274304999997, 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068647313000000001, 2024-09-14T07:00:22.528 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45774 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668062+0000", 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668064+0000", 2024-09-14T07:00:22.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.058e-06 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668070+0000", 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668075+0000", 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7859999999999996e-06 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736553+0000", 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068478400999999994 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736622+0000", 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8732999999999994e-05 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736709+0000", 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6453999999999996e-05 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.787254046999998, 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:00:22.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.786037026000002, 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:00:22.537 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:22.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:00:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.785909185999998, 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.542 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:00:22.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.785889093000002, 2024-09-14T07:00:22.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:00:22.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.784862199999999, 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:00:22.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:22.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:22.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:00:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:00:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:22.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:22.555 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:22.556 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:22.557 INFO:teuthology.orchestra.run.smithi007.stdout:flattening image image_client.0-clone22 2024-09-14T07:00:22.557 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:22.555+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:22.614 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-14T07:00:22.657 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:22.656+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:22.668 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:22.669 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:22.669 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:22.669 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:22.681 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:22.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:22.761 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:22.761+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:22.801 INFO:teuthology.orchestra.run.smithi007.stdout:3352 punch from 0x2012ca7 to 0x201efab, (0xc304 bytes) 2024-09-14T07:00:22.805 INFO:teuthology.orchestra.run.smithi007.stdout:3353 read 0x35e1051 thru 0x35f0d3d (0xfced bytes) 2024-09-14T07:00:22.806 INFO:teuthology.orchestra.run.smithi007.stdout:3355 punch from 0xb45ec8 to 0xb51548, (0xb680 bytes) 2024-09-14T07:00:22.822 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:22.822 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:22.822 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:22.823 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:22.823 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:22.824 INFO:teuthology.orchestra.run.smithi007.stdout:3356 write 0xe9f7e3c thru 0xea07515 (0xf6da bytes) 2024-09-14T07:00:22.828 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:22.828 INFO:teuthology.orchestra.run.smithi007.stdout:3359 read 0x5fd52dd thru 0x5fd58a8 (0x5cc bytes) 2024-09-14T07:00:22.828 INFO:teuthology.orchestra.run.smithi007.stdout:3360 write 0x3380542 thru 0x3386922 (0x63e1 bytes) 2024-09-14T07:00:22.831 INFO:teuthology.orchestra.run.smithi007.stdout:3361 write 0xaf76901 thru 0xaf79f2c (0x362c bytes) 2024-09-14T07:00:22.832 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:22.843 INFO:teuthology.orchestra.run.smithi007.stdout:3362 trunc from 0xea07516 to 0x6b06868 2024-09-14T07:00:22.852 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:22.862 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:22.862 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:22.867 INFO:teuthology.orchestra.run.smithi007.stdout:3365 write 0xe28261a thru 0xe28e1db (0xbbc2 bytes) 2024-09-14T07:00:22.874 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:22.873+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:22.877 INFO:teuthology.orchestra.run.smithi007.stdout:3366 trunc from 0xe28e1dc to 0x77bb4ae 2024-09-14T07:00:22.882 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:22.882 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:22.888 INFO:teuthology.orchestra.run.smithi007.stdout:3367 read 0x1a1aff4 thru 0x1a1c7de (0x17eb bytes) 2024-09-14T07:00:22.888 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:22.897 INFO:teuthology.orchestra.run.smithi007.stdout:3369 write 0xc0810e1 thru 0xc088616 (0x7536 bytes) 2024-09-14T07:00:22.900 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:22.900 INFO:teuthology.orchestra.run.smithi007.stdout:3371 write 0x282cab0 thru 0x282ea8d (0x1fde bytes) 2024-09-14T07:00:22.903 INFO:teuthology.orchestra.run.smithi007.stdout:3372 write 0x816fe4f thru 0x817b14c (0xb2fe bytes) 2024-09-14T07:00:22.914 INFO:teuthology.orchestra.run.smithi007.stdout:3373 trunc from 0xc088617 to 0x468ce51 2024-09-14T07:00:22.919 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:22.919 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:22.923 INFO:teuthology.orchestra.run.smithi007.stdout:3376 read 0x193be4c thru 0x193c8e3 (0xa98 bytes) 2024-09-14T07:00:22.924 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:22.924 INFO:teuthology.orchestra.run.smithi007.stdout:3379 writesame 0x3aee66f thru 0x3af09bc data_size 0x1(0x234e bytes) 2024-09-14T07:00:22.929 INFO:teuthology.orchestra.run.smithi007.stdout:3380 writesame 0xe2e294 thru 0xe31872 data_size 0x1(0x35df bytes) 2024-09-14T07:00:22.937 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:22.937 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:3382 punch from 0x16b1b02 to 0x16b4440, (0x293e bytes) 2024-09-14T07:00:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:3384 read 0x86a453 thru 0x8741c3 (0x9d71 bytes) 2024-09-14T07:00:22.938 INFO:teuthology.orchestra.run.smithi007.stdout:3385 write 0xc679c8 thru 0xc6c971 (0x4faa bytes) 2024-09-14T07:00:22.942 INFO:teuthology.orchestra.run.smithi007.stdout:3386 punch from 0x31eb78e to 0x31f0bc6, (0x5438 bytes) 2024-09-14T07:00:22.942 INFO:teuthology.orchestra.run.smithi007.stdout:3387 read 0x264a77 thru 0x2727cb (0xdd55 bytes) 2024-09-14T07:00:22.942 INFO:teuthology.orchestra.run.smithi007.stdout:3388 read 0x1892be9 thru 0x189ed2f (0xc147 bytes) 2024-09-14T07:00:22.948 INFO:teuthology.orchestra.run.smithi007.stdout:3389 write 0x4e1ef20 thru 0x4e2a521 (0xb602 bytes) 2024-09-14T07:00:22.951 INFO:teuthology.orchestra.run.smithi007.stdout:3390 read 0x3a426f8 thru 0x3a46b67 (0x4470 bytes) 2024-09-14T07:00:22.951 INFO:teuthology.orchestra.run.smithi007.stdout:3392 write 0x4617336 thru 0x4620299 (0x8f64 bytes) 2024-09-14T07:00:22.954 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:22.957 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:22.957 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:22.959 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:22.959+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:22.964 INFO:teuthology.orchestra.run.smithi007.stdout:3396 trunc from 0x4e2a522 to 0xd7772fd 2024-09-14T07:00:22.969 INFO:teuthology.orchestra.run.smithi007.stdout:3397 punch from 0x788ea20 to 0x7895204, (0x67e4 bytes) 2024-09-14T07:00:22.969 INFO:teuthology.orchestra.run.smithi007.stdout:3399 write 0xfb98ca thru 0xfbe8b3 (0x4fea bytes) 2024-09-14T07:00:22.973 INFO:teuthology.orchestra.run.smithi007.stdout:3400 writesame 0x1e5550a thru 0x1e55f29 data_size 0x1(0xa20 bytes) 2024-09-14T07:00:22.976 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:22.976 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:22.977 INFO:teuthology.orchestra.run.smithi007.stdout:3401 writesame 0xb5870fe thru 0xb591e30 data_size 0x1(0xad33 bytes) 2024-09-14T07:00:22.980 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:22.981 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:22.990 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:22.993 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:22.999 INFO:teuthology.orchestra.run.smithi007.stdout:3403 write 0xa9492a thru 0xa9b9b8 (0x708f bytes) 2024-09-14T07:00:23.003 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:23.003 INFO:teuthology.orchestra.run.smithi007.stdout:3405 read 0x26da78a thru 0x26e3218 (0x8a8f bytes) 2024-09-14T07:00:23.003 INFO:teuthology.orchestra.run.smithi007.stdout:3406 punch from 0x5cbcd53 to 0x5cc4da1, (0x804e bytes) 2024-09-14T07:00:23.004 INFO:teuthology.orchestra.run.smithi007.stdout:3407 read 0x3dba5e5 thru 0x3dbe5e0 (0x3ffc bytes) 2024-09-14T07:00:23.004 INFO:teuthology.orchestra.run.smithi007.stdout:3409 write 0x70cd5a8 thru 0x70cdd80 (0x7d9 bytes) 2024-09-14T07:00:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:3410 punch from 0x4efd6af to 0x4efec12, (0x1563 bytes) 2024-09-14T07:00:23.006 INFO:teuthology.orchestra.run.smithi007.stdout:3411 read 0x109349c thru 0x10a1a90 (0xe5f5 bytes) 2024-09-14T07:00:23.020 INFO:teuthology.orchestra.run.smithi007.stdout:3412 trunc from 0xd7772fd to 0x10e8aef 2024-09-14T07:00:23.043 INFO:teuthology.orchestra.run.smithi007.stdout:3413 read 0xff81d9 thru 0xffc6e6 (0x450e bytes) 2024-09-14T07:00:23.060 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:23.059+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:23.061 INFO:teuthology.orchestra.run.smithi007.stdout:3414 writesame 0xa06ccfe thru 0xa0787a4 data_size 0x1(0xbaa7 bytes) 2024-09-14T07:00:23.061 INFO:teuthology.orchestra.run.smithi007.stdout:3415 write 0x3d1c4a9 thru 0x3d26d9f (0xa8f7 bytes) 2024-09-14T07:00:23.065 INFO:teuthology.orchestra.run.smithi007.stdout:3416 write 0x4b7109c thru 0x4b7a3e7 (0x934c bytes) 2024-09-14T07:00:23.068 INFO:teuthology.orchestra.run.smithi007.stdout:3417 write 0x79bf3e9 thru 0x79c7c02 (0x881a bytes) 2024-09-14T07:00:23.072 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:23.080 INFO:teuthology.orchestra.run.smithi007.stdout:3420 write 0xccd24b4 thru 0xccd873c (0x6289 bytes) 2024-09-14T07:00:23.083 INFO:teuthology.orchestra.run.smithi007.stdout:3421 punch from 0xc41d055 to 0xc42bbd6, (0xeb81 bytes) 2024-09-14T07:00:23.083 INFO:teuthology.orchestra.run.smithi007.stdout:3423 read 0x13289cb thru 0x132a77c (0x1db2 bytes) 2024-09-14T07:00:23.083 INFO:teuthology.orchestra.run.smithi007.stdout:3425 clone 24 order 20 su 524288 sc 13 2024-09-14T07:00:23.100 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:23.100 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:23.109 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:23.161 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:23.161+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:23.213 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:23.213 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:23.221 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:23.265 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:23.264+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:23.365 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:23.365+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:23.367 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:23.367 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:23.376 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:23.467 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:23.466+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:23.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:00:23.522 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:23.523 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:23.532 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:23.559 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone23 from 0xccd873d (overlap 0x10e8aef) to 0x5129ae 2024-09-14T07:00:23.575 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:23.574+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:23.622 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-14T07:00:23.641 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:23.641 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:23.650 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:23.668 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:23.668+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:23.724 INFO:teuthology.orchestra.run.smithi007.stdout:3427 trunc from 0xccd873d to 0xca79de9 2024-09-14T07:00:23.744 INFO:teuthology.orchestra.run.smithi007.stdout:3428 trunc from 0xca79de9 to 0xa4f02ad 2024-09-14T07:00:23.762 INFO:teuthology.orchestra.run.smithi007.stdout:3430 write 0x498e0b3 thru 0x4995b31 (0x7a7f bytes) 2024-09-14T07:00:23.769 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:23.768+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:23.773 INFO:teuthology.orchestra.run.smithi007.stdout:3432 trunc from 0xa4f02ad to 0x3b4a5b7 2024-09-14T07:00:23.802 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:23.803 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:23.803 INFO:teuthology.orchestra.run.smithi007.stdout:3433 punch from 0x1c3dcf8 to 0x1c43e96, (0x619e bytes) 2024-09-14T07:00:23.811 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:23.816 INFO:teuthology.orchestra.run.smithi007.stdout:3435 trunc from 0x3b4a5b7 to 0xe963588 2024-09-14T07:00:23.822 INFO:teuthology.orchestra.run.smithi007.stdout:3436 punch from 0x598ff61 to 0x599f7b4, (0xf853 bytes) 2024-09-14T07:00:23.837 INFO:teuthology.orchestra.run.smithi007.stdout:3437 trunc from 0xe963588 to 0x12b1e24 2024-09-14T07:00:23.870 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:23.869+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:23.886 INFO:teuthology.orchestra.run.smithi007.stdout:3438 writesame 0xe6c3803 thru 0xe6c8d45 data_size 0x17(0x5543 bytes) 2024-09-14T07:00:23.886 INFO:teuthology.orchestra.run.smithi007.stdout:3439 punch from 0x8383b04 to 0x83891d0, (0x56cc bytes) 2024-09-14T07:00:23.886 INFO:teuthology.orchestra.run.smithi007.stdout:3440 writesame 0x20aa92b thru 0x20aaf49 data_size 0x1(0x61f bytes) 2024-09-14T07:00:23.889 INFO:teuthology.orchestra.run.smithi007.stdout:3441 write 0x1ea45f7 thru 0x1eab47c (0x6e86 bytes) 2024-09-14T07:00:23.892 INFO:teuthology.orchestra.run.smithi007.stdout:3442 read 0xbe0376 thru 0xbee53d (0xe1c8 bytes) 2024-09-14T07:00:23.892 INFO:teuthology.orchestra.run.smithi007.stdout:3443 write 0xa6f52db thru 0xa6f9d1e (0x4a44 bytes) 2024-09-14T07:00:23.895 INFO:teuthology.orchestra.run.smithi007.stdout:3448 writesame 0x3f59796 thru 0x3f6788f data_size 0x57(0xe0fa bytes) 2024-09-14T07:00:23.898 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:23.914 INFO:teuthology.orchestra.run.smithi007.stdout:3450 trunc from 0xe6c8d46 to 0x41dfce 2024-09-14T07:00:23.947 INFO:teuthology.orchestra.run.smithi007.stdout:3451 writesame 0x173581e thru 0x1738806 data_size 0xb(0x2fe9 bytes) 2024-09-14T07:00:23.954 INFO:teuthology.orchestra.run.smithi007.stdout:3452 write 0x31d281e thru 0x31e1f2a (0xf70d bytes) 2024-09-14T07:00:23.958 INFO:teuthology.orchestra.run.smithi007.stdout:3453 punch from 0x2c7aade to 0x2c7cd87, (0x22a9 bytes) 2024-09-14T07:00:23.959 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:23.959 INFO:teuthology.orchestra.run.smithi007.stdout:3455 read 0x4159e8 thru 0x41ede1 (0x93fa bytes) 2024-09-14T07:00:23.962 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:23.962 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:23.963 INFO:teuthology.orchestra.run.smithi007.stdout:3456 trunc from 0x31e1f2b to 0x6c98412 2024-09-14T07:00:23.968 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:23.970 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:23.970+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:23.971 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:23.975 INFO:teuthology.orchestra.run.smithi007.stdout:3460 write 0x7d94fa5 thru 0x7d9c9e0 (0x7a3c bytes) 2024-09-14T07:00:23.980 INFO:teuthology.orchestra.run.smithi007.stdout:3461 trunc from 0x7d9c9e1 to 0x87eaea4 2024-09-14T07:00:23.988 INFO:teuthology.orchestra.run.smithi007.stdout:3463 trunc from 0x87eaea4 to 0xb139d2e 2024-09-14T07:00:23.993 INFO:teuthology.orchestra.run.smithi007.stdout:3464 read 0x331bc04 thru 0x3329179 (0xd576 bytes) 2024-09-14T07:00:23.999 INFO:teuthology.orchestra.run.smithi007.stdout:3465 write 0xb94825e thru 0xb94c3a9 (0x414c bytes) 2024-09-14T07:00:24.004 INFO:teuthology.orchestra.run.smithi007.stdout:3466 write 0x4726596 thru 0x473218c (0xbbf7 bytes) 2024-09-14T07:00:24.010 INFO:teuthology.orchestra.run.smithi007.stdout:3467 trunc from 0xb94c3aa to 0xd005e87 2024-09-14T07:00:24.014 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.014 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.015 INFO:teuthology.orchestra.run.smithi007.stdout:3471 write 0x87eb0e5 thru 0x87f4d86 (0x9ca2 bytes) 2024-09-14T07:00:24.018 INFO:teuthology.orchestra.run.smithi007.stdout:3472 write 0xcd6b663 thru 0xcd6f31d (0x3cbb bytes) 2024-09-14T07:00:24.021 INFO:teuthology.orchestra.run.smithi007.stdout:3474 clone 25 order 24 su 524288 sc 8 2024-09-14T07:00:24.072 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:24.071+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:24.116 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:24.116 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:24.124 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:24.209 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:24.208+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:24.268 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:24.268 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:24.274 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:24.274+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:24.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:24.375 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:24.375+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:24.386 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:24.386 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:24.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:24.477 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:24.476+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:24.504 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T07:00:24.507 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:24.507 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:24.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:24.571 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone24 from 0xd005e87 (overlap 0x41dfce) to 0x0 2024-09-14T07:00:24.578 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:24.577+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:24.632 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:24.632 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:24.634 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-14T07:00:24.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:24.644 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.644 INFO:teuthology.orchestra.run.smithi007.stdout:3476 write 0x959cd3 thru 0x96141e (0x774c bytes) 2024-09-14T07:00:24.650 INFO:teuthology.orchestra.run.smithi007.stdout:3477 write 0x3cf79c2 thru 0x3d047da (0xce19 bytes) 2024-09-14T07:00:24.656 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.667 INFO:teuthology.orchestra.run.smithi007.stdout:3479 trunc from 0xd005e87 to 0x36c3de5 2024-09-14T07:00:24.679 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:24.678+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:24.683 INFO:teuthology.orchestra.run.smithi007.stdout:3480 read 0x35ab543 thru 0x35af140 (0x3bfe bytes) 2024-09-14T07:00:24.683 INFO:teuthology.orchestra.run.smithi007.stdout:3481 read 0xa8d99a thru 0xa9d101 (0xf768 bytes) 2024-09-14T07:00:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:3482 write 0x9474c9c thru 0x947a08f (0x53f4 bytes) 2024-09-14T07:00:24.698 INFO:teuthology.orchestra.run.smithi007.stdout:3483 read 0x8983c7 thru 0x8a6376 (0xdfb0 bytes) 2024-09-14T07:00:24.698 INFO:teuthology.orchestra.run.smithi007.stdout:3484 write 0x848fc8f thru 0x84960b8 (0x642a bytes) 2024-09-14T07:00:24.701 INFO:teuthology.orchestra.run.smithi007.stdout:3485 read 0x3928acf thru 0x3933bc4 (0xb0f6 bytes) 2024-09-14T07:00:24.702 INFO:teuthology.orchestra.run.smithi007.stdout:3486 read 0x872ef9a thru 0x873ecc2 (0xfd29 bytes) 2024-09-14T07:00:24.702 INFO:teuthology.orchestra.run.smithi007.stdout:3487 write 0x2b1bc12 thru 0x2b28977 (0xcd66 bytes) 2024-09-14T07:00:24.710 INFO:teuthology.orchestra.run.smithi007.stdout:3488 write 0xa0fab05 thru 0xa102ebc (0x83b8 bytes) 2024-09-14T07:00:24.713 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.718 INFO:teuthology.orchestra.run.smithi007.stdout:3493 trunc from 0xa102ebd to 0x53a5de4 2024-09-14T07:00:24.744 INFO:teuthology.orchestra.run.smithi007.stdout:3494 write 0xbffdb40 thru 0xc009e65 (0xc326 bytes) 2024-09-14T07:00:24.748 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.748 INFO:teuthology.orchestra.run.smithi007.stdout:3496 writesame 0x325d641 thru 0x3264c62 data_size 0x1(0x7622 bytes) 2024-09-14T07:00:24.764 INFO:teuthology.orchestra.run.smithi007.stdout:3498 writesame 0x79af7d8 thru 0x79b20a7 data_size 0x8(0x28d0 bytes) 2024-09-14T07:00:24.766 INFO:teuthology.orchestra.run.smithi007.stdout:3499 read 0x2fe9871 thru 0x2fea8d6 (0x1066 bytes) 2024-09-14T07:00:24.766 INFO:teuthology.orchestra.run.smithi007.stdout:3502 writesame 0x7d84228 thru 0x7d89b4a data_size 0x1(0x5923 bytes) 2024-09-14T07:00:24.776 INFO:teuthology.orchestra.run.smithi007.stdout:3503 read 0x765683e thru 0x7657e09 (0x15cc bytes) 2024-09-14T07:00:24.777 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.780 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:24.779+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:24.780 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:24.781 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:24.790 INFO:teuthology.orchestra.run.smithi007.stdout:3505 trunc from 0xc009e66 to 0x9e8313 2024-09-14T07:00:24.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:24.803 INFO:teuthology.orchestra.run.smithi007.stdout:3507 flatten 2024-09-14T07:00:24.809 INFO:teuthology.orchestra.run.smithi007.stdout:3510 read 0x4e032d thru 0x4e804b (0x7d1f bytes) 2024-09-14T07:00:24.825 INFO:teuthology.orchestra.run.smithi007.stdout:3512 trunc from 0x9e8313 to 0xe91a03b 2024-09-14T07:00:24.830 INFO:teuthology.orchestra.run.smithi007.stdout:3513 write 0x461a353 thru 0x461d649 (0x32f7 bytes) 2024-09-14T07:00:24.832 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.832 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.832 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.832 INFO:teuthology.orchestra.run.smithi007.stdout:3519 writesame 0x6be8b1a thru 0x6bf03e9 data_size 0x8(0x78d0 bytes) 2024-09-14T07:00:24.835 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.835 INFO:teuthology.orchestra.run.smithi007.stdout:3522 write 0x15cf79f thru 0x15db42d (0xbc8f bytes) 2024-09-14T07:00:24.836 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.837 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.837 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.837 INFO:teuthology.orchestra.run.smithi007.stdout:3526 read 0x211affa thru 0x21265f9 (0xb600 bytes) 2024-09-14T07:00:24.837 INFO:teuthology.orchestra.run.smithi007.stdout:3527 write 0xdfcc84a thru 0xdfda3c2 (0xdb79 bytes) 2024-09-14T07:00:24.841 INFO:teuthology.orchestra.run.smithi007.stdout:3528 punch from 0x6efdccf to 0x6efe6ba, (0x9eb bytes) 2024-09-14T07:00:24.841 INFO:teuthology.orchestra.run.smithi007.stdout:3529 punch from 0x79888a9 to 0x798b943, (0x309a bytes) 2024-09-14T07:00:24.841 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.841 INFO:teuthology.orchestra.run.smithi007.stdout:3532 read 0xa3cfaa4 thru 0xa3d2781 (0x2cde bytes) 2024-09-14T07:00:24.841 INFO:teuthology.orchestra.run.smithi007.stdout:3533 read 0x7dec7c thru 0x7e61f8 (0x757d bytes) 2024-09-14T07:00:24.842 INFO:teuthology.orchestra.run.smithi007.stdout:3534 read 0x8d699f5 thru 0x8d6dd48 (0x4354 bytes) 2024-09-14T07:00:24.842 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:24.842 INFO:teuthology.orchestra.run.smithi007.stdout:3537 punch from 0x7fe54b5 to 0x7ff1c3e, (0xc789 bytes) 2024-09-14T07:00:24.842 INFO:teuthology.orchestra.run.smithi007.stdout:3538 writesame 0x4e85b2a thru 0x4e913a2 data_size 0x5(0xb879 bytes) 2024-09-14T07:00:24.846 INFO:teuthology.orchestra.run.smithi007.stdout:3539 read 0xcb9f9b1 thru 0xcba00e2 (0x732 bytes) 2024-09-14T07:00:24.847 INFO:teuthology.orchestra.run.smithi007.stdout:3542 punch from 0xb362e30 to 0xb36fbcd, (0xcd9d bytes) 2024-09-14T07:00:24.847 INFO:teuthology.orchestra.run.smithi007.stdout:3543 write 0x7eb6a01 thru 0x7ec5bd1 (0xf1d1 bytes) 2024-09-14T07:00:24.849 INFO:teuthology.orchestra.run.smithi007.stdout:3544 punch from 0x3f6f2a5 to 0x3f790d7, (0x9e32 bytes) 2024-09-14T07:00:24.849 INFO:teuthology.orchestra.run.smithi007.stdout:3545 clone 26 order 18 su 131072 sc 8 2024-09-14T07:00:24.880 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:24.880+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:24.915 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:24.915 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:24.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:25.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:25.009 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:25.008+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:25.072 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:25.072 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:25.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:25.082 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:25.082+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:25.135 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:25.152 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:25.153 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:25.169 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:25.169 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:25.183 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:25.183+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:25.186 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:25.187 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:25.192 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:25.192 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:25.201 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:25.204 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:25.205 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:25.222 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:25.222 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:25.240 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:25.240 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:25.257 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:25.257 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:25.285 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:25.284+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:25.312 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:25.312 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:25.313 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:25.313 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:25.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:25.385 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:25.385+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:25.430 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119746 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1298432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.88393947, 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067798875999999994, 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119746 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545019+0000", 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545020+0000", 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.782e-06 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545026+0000", 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9390000000000002e-06 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545031+0000", 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5770000000000004e-06 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612601+0000", 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067570682000000007 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612683+0000", 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1378999999999995e-05 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612817+0000", 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013415000000000001 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119747 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.883921051, 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068036012000000007, 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119747 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546792+0000", 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017554960000000001 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546797+0000", 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1950000000000002e-06 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546808+0000", 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0675e-05 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546815+0000", 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612854+0000", 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066039158000000001 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612930+0000", 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.613073+0000", 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000142798 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119753 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1398784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.881996889, 2024-09-14T07:00:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067577770999999995, 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119753 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546993+0000", 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1860999999999999e-05 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547007+0000", 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4346e-05 2024-09-14T07:00:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547016+0000", 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4709999999999996e-06 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547023+0000", 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4010000000000002e-06 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614322+0000", 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067299231000000001 2024-09-14T07:00:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614391+0000", 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8826999999999997e-05 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614538+0000", 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000147634 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119754 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1413120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.881906351, 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067735654000000006, 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119754 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549973+0000", 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029215999999999999 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549978+0000", 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3510000000000001e-06 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549988+0000", 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0037000000000001e-05 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549995+0000", 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9709999999999998e-06 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614569+0000", 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064573368000000006 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614650+0000", 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1909000000000005e-05 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614787+0000", 2024-09-14T07:00:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013641799999999999 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119764 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1556480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.878734443999999, 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067672107999999995, 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119764 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550224+0000", 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550226+0000", 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1720000000000001e-06 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550233+0000", 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9879999999999998e-06 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550238+0000", 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1819999999999997e-06 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617715+0000", 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067476628999999996 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617807+0000", 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2899999999999995e-05 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617895+0000", 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7837999999999995e-05 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119765 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.878712011999999, 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067829500000000001, 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119765 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7e-07 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550248+0000", 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1280000000000002e-06 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550256+0000", 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550261+0000", 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617926+0000", 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067665280999999994 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617995+0000", 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9478000000000001e-05 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618075+0000", 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9666000000000006e-05 2024-09-14T07:00:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119950 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 529408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.878689906, 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067992002999999995, 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119950 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550852+0000", 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058435400000000004 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550857+0000", 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6979999999999997e-06 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550867+0000", 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0378e-05 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550874+0000", 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8480000000000003e-06 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618106+0000", 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067231714999999997 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618178+0000", 2024-09-14T07:00:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2572999999999998e-05 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618260+0000", 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1437e-05 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119970 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 816128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.872034749999999, 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067745828999999994, 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119970 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556924+0000", 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0410000000000001e-06 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556927+0000", 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9679999999999998e-06 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556930+0000", 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624487+0000", 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067557392999999993 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624551+0000", 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3861000000000004e-05 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624669+0000", 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117921 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119971 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.872024810999999, 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067869065000000006, 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119971 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-14T07:00:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556934+0000", 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.096e-06 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556937+0000", 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.796e-06 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556939+0000", 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624708+0000", 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067768992 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624747+0000", 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8148999999999997e-05 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624802+0000", 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.533e-05 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119972 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 844800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.87201514, 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068010762000000002, 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119972 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556943+0000", 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556944+0000", 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556947+0000", 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9720000000000001e-06 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556949+0000", 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4899999999999999e-06 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624838+0000", 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067888899000000003 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624895+0000", 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7395000000000002e-05 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624953+0000", 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7745999999999999e-05 2024-09-14T07:00:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119973 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 859136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.872005258, 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068210898000000006, 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119973 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556953+0000", 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556954+0000", 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189999999999999e-06 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556957+0000", 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9490000000000001e-06 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556959+0000", 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3870000000000002e-06 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624983+0000", 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068023726000000007 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625049+0000", 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6385999999999998e-05 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625163+0000", 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114221 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119974 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:25.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.871995337, 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068338594000000003, 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119974 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:25.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556963+0000", 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556964+0000", 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556969+0000", 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.04e-06 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556974+0000", 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.639e-06 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625194+0000", 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068220742000000001 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625246+0000", 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1844e-05 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625301+0000", 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4551e-05 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119975 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.871977097, 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068554198999999996, 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119975 2024-09-14T07:00:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556983+0000", 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2560000000000001e-06 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556989+0000", 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2210000000000002e-06 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556994+0000", 2024-09-14T07:00:25.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7779999999999998e-06 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625330+0000", 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068335911999999999 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625396+0000", 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6051999999999997e-05 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625535+0000", 2024-09-14T07:00:25.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013860200000000001 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119976 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.871956102, 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068656164000000006, 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119976 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557004+0000", 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557010+0000", 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1770000000000003e-06 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557016+0000", 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2650000000000004e-06 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625566+0000", 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068550323999999996 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625604+0000", 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8398000000000001e-05 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625658+0000", 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3239000000000003e-05 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119977 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.871875683000001, 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068713522999999999, 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119977 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:25.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557086+0000", 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3569999999999998e-06 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557094+0000", 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6829999999999997e-06 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557101+0000", 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625689+0000", 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068587882000000003 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625743+0000", 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4082000000000002e-05 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625795+0000", 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2657000000000001e-05 2024-09-14T07:00:25.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119978 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 930816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.871847090999999, 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068826410000000005, 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119978 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557113+0000", 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557120+0000", 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1079999999999998e-06 2024-09-14T07:00:25.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557125+0000", 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625823+0000", 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068698928000000006 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625884+0000", 2024-09-14T07:00:25.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0056999999999998e-05 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625937+0000", 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3383e-05 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119979 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 945152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.871825943999999, 2024-09-14T07:00:25.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068940788000000003, 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119979 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5199999999999998e-07 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557134+0000", 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.018e-06 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557140+0000", 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557145+0000", 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7790000000000003e-06 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625966+0000", 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068820972999999994 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626019+0000", 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3828e-05 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626072+0000", 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3078000000000002e-05 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119980 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.871805586000001, 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069053630000000005, 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119980 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557159+0000", 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557167+0000", 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999996e-06 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557172+0000", 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-14T07:00:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626101+0000", 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068928552000000004 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626154+0000", 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3338000000000003e-05 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626206+0000", 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1657999999999999e-05 2024-09-14T07:00:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119981 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.871776936, 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069157629999999998, 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119981 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557183+0000", 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557190+0000", 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2249999999999997e-06 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557195+0000", 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3120000000000001e-06 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626234+0000", 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069038926 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626286+0000", 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2604e-05 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626338+0000", 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1947000000000003e-05 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119982 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 988160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.871755782999999, 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069364004000000007, 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119982 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560989+0000", 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037867360000000002 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560994+0000", 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561004+0000", 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3680000000000008e-06 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561011+0000", 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1640000000000004e-06 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626366+0000", 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.065355286999999998 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626433+0000", 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6554e-05 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626566+0000", 2024-09-14T07:00:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132996 2024-09-14T07:00:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.518 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:25.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:25.520 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:25.518+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:25.602 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7] out_osds: [6, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:25.602 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:25.602 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-14T07:00:25.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-14T07:00:25.648 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:25.647+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:25.674 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:25.674 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:25.674 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:25.674 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:25.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:25.705 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-14T07:00:25.707 INFO:teuthology.orchestra.run.smithi007.stdout:3546 read 0x63fc34d thru 0x640ad9c (0xea50 bytes) 2024-09-14T07:00:25.723 INFO:teuthology.orchestra.run.smithi007.stdout:3547 trunc from 0xe91a03b to 0x7e7e81 2024-09-14T07:00:25.723 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:25.723+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:25.737 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:25.750 INFO:teuthology.orchestra.run.smithi007.stdout:3550 trunc from 0x7e7e81 to 0xbb1bdd1 2024-09-14T07:00:25.757 INFO:teuthology.orchestra.run.smithi007.stdout:3551 punch from 0x5cea5ca to 0x5cf68a7, (0xc2dd bytes) 2024-09-14T07:00:25.759 INFO:teuthology.orchestra.run.smithi007.stdout:3552 trunc from 0xbb1bdd1 to 0xd39e63a 2024-09-14T07:00:25.764 INFO:teuthology.orchestra.run.smithi007.stdout:3553 read 0x34f2d78 thru 0x34ffaa1 (0xcd2a bytes) 2024-09-14T07:00:25.764 INFO:teuthology.orchestra.run.smithi007.stdout:3554 write 0x19cc394 thru 0x19dbb83 (0xf7f0 bytes) 2024-09-14T07:00:25.770 INFO:teuthology.orchestra.run.smithi007.stdout:3555 trunc from 0xd39e63a to 0xe3d37f5 2024-09-14T07:00:25.775 INFO:teuthology.orchestra.run.smithi007.stdout:3556 write 0x3bd28a4 thru 0x3bd321d (0x97a bytes) 2024-09-14T07:00:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:3557 punch from 0x78e270 to 0x78e92c, (0x6bc bytes) 2024-09-14T07:00:25.779 INFO:teuthology.orchestra.run.smithi007.stdout:3558 writesame 0x40afa3b thru 0x40b714a data_size 0x50(0x7710 bytes) 2024-09-14T07:00:25.782 INFO:teuthology.orchestra.run.smithi007.stdout:3559 read 0x7dd1d84 thru 0x7dda00e (0x828b bytes) 2024-09-14T07:00:25.797 INFO:teuthology.orchestra.run.smithi007.stdout:3560 trunc from 0xe3d37f5 to 0x8cc242 2024-09-14T07:00:25.808 INFO:teuthology.orchestra.run.smithi007.stdout:3561 write 0x694fd thru 0x77bc8 (0xe6cc bytes) 2024-09-14T07:00:25.813 INFO:teuthology.orchestra.run.smithi007.stdout:3563 writesame 0x3d3e7 thru 0x4b048 data_size 0x3(0xdc62 bytes) 2024-09-14T07:00:25.821 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:25.821+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:25.832 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:25.833 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:25.837 INFO:teuthology.orchestra.run.smithi007.stdout:3564 write 0x44bb957 thru 0x44c45ac (0x8c56 bytes) 2024-09-14T07:00:25.841 INFO:teuthology.orchestra.run.smithi007.stdout:3565 read 0x3f04f77 thru 0x3f0c066 (0x70f0 bytes) 2024-09-14T07:00:25.841 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:25.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:25.855 INFO:teuthology.orchestra.run.smithi007.stdout:3567 writesame 0xb36e4a7 thru 0xb37c0bb data_size 0x1(0xdc15 bytes) 2024-09-14T07:00:25.855 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:25.855 INFO:teuthology.orchestra.run.smithi007.stdout:3571 writesame 0x2a0103b thru 0x2a0e5d1 data_size 0x1(0xd597 bytes) 2024-09-14T07:00:25.890 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:25.890 INFO:teuthology.orchestra.run.smithi007.stdout:3573 write 0x923bb61 thru 0x923ffdc (0x447c bytes) 2024-09-14T07:00:25.893 INFO:teuthology.orchestra.run.smithi007.stdout:3574 write 0xe2baad thru 0xe3563b (0x9b8f bytes) 2024-09-14T07:00:25.904 INFO:teuthology.orchestra.run.smithi007.stdout:3576 trunc from 0xb37c0bc to 0x5279258 2024-09-14T07:00:25.923 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:25.922+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:25.929 INFO:teuthology.orchestra.run.smithi007.stdout:3578 writesame 0xb4005ca thru 0xb404967 data_size 0xf(0x439e bytes) 2024-09-14T07:00:25.929 INFO:teuthology.orchestra.run.smithi007.stdout:3579 write 0x2cf2a22 thru 0x2d025ba (0xfb99 bytes) 2024-09-14T07:00:25.933 INFO:teuthology.orchestra.run.smithi007.stdout:3580 write 0x49fcc97 thru 0x4a08015 (0xb37f bytes) 2024-09-14T07:00:25.937 INFO:teuthology.orchestra.run.smithi007.stdout:3581 punch from 0x5ac5c9 to 0x5b4d00, (0x8737 bytes) 2024-09-14T07:00:25.938 INFO:teuthology.orchestra.run.smithi007.stdout:3582 writesame 0x53aec7d thru 0x53b6e3d data_size 0x1(0x81c1 bytes) 2024-09-14T07:00:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:3583 read 0x5988c28 thru 0x5995e88 (0xd261 bytes) 2024-09-14T07:00:25.957 INFO:teuthology.orchestra.run.smithi007.stdout:3584 read 0x9a3196c thru 0x9a37a40 (0x60d5 bytes) 2024-09-14T07:00:25.965 INFO:teuthology.orchestra.run.smithi007.stdout:3585 write 0xeae18f7 thru 0xeae76f5 (0x5dff bytes) 2024-09-14T07:00:25.969 INFO:teuthology.orchestra.run.smithi007.stdout:3588 trunc from 0xeae76f6 to 0xe01b28c 2024-09-14T07:00:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:3589 read 0x257ff2a thru 0x258e110 (0xe1e7 bytes) 2024-09-14T07:00:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:3590 punch from 0xcd38571 to 0xcd3f21f, (0x6cae bytes) 2024-09-14T07:00:25.978 INFO:teuthology.orchestra.run.smithi007.stdout:3591 write 0x6c0a71e thru 0x6c112ba (0x6b9d bytes) 2024-09-14T07:00:25.981 INFO:teuthology.orchestra.run.smithi007.stdout:3592 write 0x244db06 thru 0x2451e3d (0x4338 bytes) 2024-09-14T07:00:25.983 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-14T07:00:25.982+0000 7fbb91e53640 -1 mon.a@0(leader).osd e72 definitely_dead 0 2024-09-14T07:00:25.984 INFO:teuthology.orchestra.run.smithi007.stdout:3593 writesame 0x75a11fa thru 0x75a3de3 data_size 0x49(0x2bea bytes) 2024-09-14T07:00:25.987 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:00:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:25.987 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.231770961999999, 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:00:25.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:00:25.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:25.994 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.633358370000003, 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:25.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:00:25.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:00:25.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:25.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.627698727000002, 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:00:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.321862449000001, 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:00:26.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:00:26.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.321831656999997, 2024-09-14T07:00:26.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:26.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:00:26.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.321808287000003, 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:00:26.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.021 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.321001778000003, 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:00:26.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:00:26.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:26.026 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.320979385000001, 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:00:26.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:00:26.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.320959244999997, 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.031 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.320809089999997, 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:00:26.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:00:26.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.320753535999998, 2024-09-14T07:00:26.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.320720495000003, 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:00:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.31836741, 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:00:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:00:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:00:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:00:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.318344844999999, 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:00:26.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:00:26.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45774 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 933888~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:26.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.317508107999998, 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068647313000000001, 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45774 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:26.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668061+0000", 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668062+0000", 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668064+0000", 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.058e-06 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668070+0000", 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668075+0000", 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7859999999999996e-06 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736553+0000", 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068478400999999994 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736622+0000", 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8732999999999994e-05 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736709+0000", 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6453999999999996e-05 2024-09-14T07:00:26.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.317487849999999, 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:00:26.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:00:26.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.316270828999997, 2024-09-14T07:00:26.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.316142988999999, 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:00:26.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:00:26.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.316122896000003, 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:00:26.076 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:00:26.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.315096003000001, 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:00:26.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:26.085 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:26.086 INFO:teuthology.orchestra.run.smithi007.stdout:3594 write 0x472db14 thru 0x4738f74 (0xb461 bytes) 2024-09-14T07:00:26.087 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:26.086+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:26.088 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:26.089 INFO:teuthology.orchestra.run.smithi007.stdout:3596 trunc from 0xe01b28c to 0x616743b 2024-09-14T07:00:26.089 INFO:teuthology.orchestra.run.smithi007.stdout:3599 write 0x1076288 thru 0x107ea98 (0x8811 bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3601 writesame 0xb787396 thru 0xb795811 data_size 0xe(0xe47c bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3603 read 0x475b627 thru 0x4761aa0 (0x647a bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3604 trunc from 0xb795812 to 0x248ca0f 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3607 trunc from 0x248ca0f to 0xd65f9ea 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3608 read 0x79d0d82 thru 0x79d6058 (0x52d7 bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3611 trunc from 0xd65f9ea to 0xd3bff5e 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3612 read 0x69b5364 thru 0x69bfa17 (0xa6b4 bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3614 writesame 0xa48c33e thru 0xa48fdbc data_size 0x1(0x3a7f bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3615 write 0xa14ee5d thru 0xa152615 (0x37b9 bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3616 write 0x5907bf4 thru 0x590f157 (0x7564 bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3617 punch from 0x4012d7 to 0x408b42, (0x786b bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3619 write 0xad55968 thru 0xad58fbe (0x3657 bytes) 2024-09-14T07:00:26.090 INFO:teuthology.orchestra.run.smithi007.stdout:3621 write 0x7818bb1 thru 0x7826700 (0xdb50 bytes) 2024-09-14T07:00:26.091 INFO:teuthology.orchestra.run.smithi007.stdout:3624 write 0x85c36a6 thru 0x85c453c (0xe97 bytes) 2024-09-14T07:00:26.093 INFO:teuthology.orchestra.run.smithi007.stdout:3628 write 0x6443fd4 thru 0x64465d2 (0x25ff bytes) 2024-09-14T07:00:26.096 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.102 INFO:teuthology.orchestra.run.smithi007.stdout:3631 write 0xdc07ac1 thru 0xdc16f93 (0xf4d3 bytes) 2024-09-14T07:00:26.105 INFO:teuthology.orchestra.run.smithi007.stdout:3632 writesame 0x4b283ee thru 0x4b3568c data_size 0x9(0xd29f bytes) 2024-09-14T07:00:26.110 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.110 INFO:teuthology.orchestra.run.smithi007.stdout:3636 read 0xfc5098 thru 0xfc9fae (0x4f17 bytes) 2024-09-14T07:00:26.110 INFO:teuthology.orchestra.run.smithi007.stdout:3637 read 0x5e198ee thru 0x5e19917 (0x2a bytes) 2024-09-14T07:00:26.110 INFO:teuthology.orchestra.run.smithi007.stdout:3638 punch from 0x5bdf01d to 0x5bea229, (0xb20c bytes) 2024-09-14T07:00:26.111 INFO:teuthology.orchestra.run.smithi007.stdout:3639 read 0x82f3ee0 thru 0x8303c39 (0xfd5a bytes) 2024-09-14T07:00:26.122 INFO:teuthology.orchestra.run.smithi007.stdout:3640 trunc from 0xdc16f94 to 0x27e0d29 2024-09-14T07:00:26.140 INFO:teuthology.orchestra.run.smithi007.stdout:3641 write 0x366f435 thru 0x36750fe (0x5cca bytes) 2024-09-14T07:00:26.143 INFO:teuthology.orchestra.run.smithi007.stdout:3643 read 0x2a8aa44 thru 0x2a93506 (0x8ac3 bytes) 2024-09-14T07:00:26.160 INFO:teuthology.orchestra.run.smithi007.stdout:3644 write 0xbbf1508 thru 0xbbf80a4 (0x6b9d bytes) 2024-09-14T07:00:26.162 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.163 INFO:teuthology.orchestra.run.smithi007.stdout:3646 punch from 0x801e7a8 to 0x802b9d1, (0xd229 bytes) 2024-09-14T07:00:26.163 INFO:teuthology.orchestra.run.smithi007.stdout:3647 write 0x53ee7b thru 0x5425b5 (0x373b bytes) 2024-09-14T07:00:26.167 INFO:teuthology.orchestra.run.smithi007.stdout:3648 write 0x85bbc15 thru 0x85c9fbd (0xe3a9 bytes) 2024-09-14T07:00:26.171 INFO:teuthology.orchestra.run.smithi007.stdout:3649 write 0x48e13e8 thru 0x48efad0 (0xe6e9 bytes) 2024-09-14T07:00:26.174 INFO:teuthology.orchestra.run.smithi007.stdout:3653 read 0xd0ab84 thru 0xd0ba39 (0xeb6 bytes) 2024-09-14T07:00:26.174 INFO:teuthology.orchestra.run.smithi007.stdout:3654 read 0xb15ea6 thru 0xb17643 (0x179e bytes) 2024-09-14T07:00:26.174 INFO:teuthology.orchestra.run.smithi007.stdout:3655 write 0x729d1f0 thru 0x72a1be5 (0x49f6 bytes) 2024-09-14T07:00:26.178 INFO:teuthology.orchestra.run.smithi007.stdout:3657 read 0x3c5c270 thru 0x3c66995 (0xa726 bytes) 2024-09-14T07:00:26.179 INFO:teuthology.orchestra.run.smithi007.stdout:3658 trunc from 0xbbf80a5 to 0xb045bcd 2024-09-14T07:00:26.187 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:26.187+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:26.189 INFO:teuthology.orchestra.run.smithi007.stdout:3659 write 0x11218c thru 0x11ae5f (0x8cd4 bytes) 2024-09-14T07:00:26.193 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.193 INFO:teuthology.orchestra.run.smithi007.stdout:3661 read 0x4a516a0 thru 0x4a576f6 (0x6057 bytes) 2024-09-14T07:00:26.199 INFO:teuthology.orchestra.run.smithi007.stdout:3662 writesame 0xba25727 thru 0xba2bb63 data_size 0x1(0x643d bytes) 2024-09-14T07:00:26.199 INFO:teuthology.orchestra.run.smithi007.stdout:3663 read 0x74ec804 thru 0x74f16e1 (0x4ede bytes) 2024-09-14T07:00:26.199 INFO:teuthology.orchestra.run.smithi007.stdout:3664 punch from 0x985ba2f to 0x985f0ec, (0x36bd bytes) 2024-09-14T07:00:26.199 INFO:teuthology.orchestra.run.smithi007.stdout:3665 write 0x129c7c5 thru 0x12a99fa (0xd236 bytes) 2024-09-14T07:00:26.202 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:26.202 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:26.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.202 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:137771 3.2 3:461ca825:::rbd_data.114f74bd3bbf.000000000000004c:head [set-alloc-hint object_size 262144 write_size 262144,write 116677~14395 in=14395b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-14T07:00:26.202 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:26.201670+0000", 2024-09-14T07:00:26.202 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.000366881, 2024-09-14T07:00:26.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038592299999999998, 2024-09-14T07:00:26.202 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for sub ops", 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 137771 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.201670+0000", 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.201670+0000", 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.201670+0000", 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7400000000000002e-07 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.201686+0000", 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5546e-05 2024-09-14T07:00:26.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.201698+0000", 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1977999999999999e-05 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.201702+0000", 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5299999999999998e-06 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.201741+0000", 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8685e-05 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.201881+0000", 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013992099999999999 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:26.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.201967+0000", 2024-09-14T07:00:26.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5699999999999996e-05 2024-09-14T07:00:26.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.207 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.207 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.207 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-14T07:00:26.207 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 1 2024-09-14T07:00:26.207 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:26.208 INFO:teuthology.orchestra.run.smithi007.stdout:3666 write 0xa7c2093 thru 0xa7c3a08 (0x1976 bytes) 2024-09-14T07:00:26.208 INFO:teuthology.orchestra.run.smithi007.stdout:3668 read 0xb07d57e thru 0xb07db25 (0x5a8 bytes) 2024-09-14T07:00:26.208 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.212 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:26.213 INFO:teuthology.orchestra.run.smithi007.stdout:3671 write 0xcca72bc thru 0xcca773d (0x482 bytes) 2024-09-14T07:00:26.216 INFO:teuthology.orchestra.run.smithi007.stdout:3672 writesame 0x2c3f4e8 thru 0x2c47dff data_size 0x29(0x8918 bytes) 2024-09-14T07:00:26.220 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.220 INFO:teuthology.orchestra.run.smithi007.stdout:3675 punch from 0x17be0b3 to 0x17c62a1, (0x81ee bytes) 2024-09-14T07:00:26.220 INFO:teuthology.orchestra.run.smithi007.stdout:3677 punch from 0x3d1bf4d to 0x3d1d13a, (0x11ed bytes) 2024-09-14T07:00:26.220 INFO:teuthology.orchestra.run.smithi007.stdout:3678 punch from 0x2a36286 to 0x2a3f26a, (0x8fe4 bytes) 2024-09-14T07:00:26.220 INFO:teuthology.orchestra.run.smithi007.stdout:3681 write 0x17aa7a3 thru 0x17ba1a8 (0xfa06 bytes) 2024-09-14T07:00:26.223 INFO:teuthology.orchestra.run.smithi007.stdout:3683 punch from 0x76154af to 0x762122f, (0xbd80 bytes) 2024-09-14T07:00:26.223 INFO:teuthology.orchestra.run.smithi007.stdout:3684 read 0xc6785b7 thru 0xc678f84 (0x9ce bytes) 2024-09-14T07:00:26.223 INFO:teuthology.orchestra.run.smithi007.stdout:3686 read 0x2485f0a thru 0x2493b75 (0xdc6c bytes) 2024-09-14T07:00:26.223 INFO:teuthology.orchestra.run.smithi007.stdout:3687 punch from 0x2ef2d02 to 0x2eff07b, (0xc379 bytes) 2024-09-14T07:00:26.230 INFO:teuthology.orchestra.run.smithi007.stdout:3689 trunc from 0xcca773e to 0x69935a3 2024-09-14T07:00:26.241 INFO:teuthology.orchestra.run.smithi007.stdout:3690 read 0x67d7bf7 thru 0x67de702 (0x6b0c bytes) 2024-09-14T07:00:26.241 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.241 INFO:teuthology.orchestra.run.smithi007.stdout:3693 read 0x46c1de thru 0x47055f (0x4382 bytes) 2024-09-14T07:00:26.241 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.243 INFO:teuthology.orchestra.run.smithi007.stdout:3696 trunc from 0x69935a3 to 0x49f8b8e 2024-09-14T07:00:26.251 INFO:teuthology.orchestra.run.smithi007.stdout:3697 trunc from 0x49f8b8e to 0x575d964 2024-09-14T07:00:26.256 INFO:teuthology.orchestra.run.smithi007.stdout:3698 read 0x1faffa9 thru 0x1fb4820 (0x4878 bytes) 2024-09-14T07:00:26.256 INFO:teuthology.orchestra.run.smithi007.stdout:3699 read 0x3338a70 thru 0x333f2a9 (0x683a bytes) 2024-09-14T07:00:26.258 INFO:teuthology.orchestra.run.smithi007.stdout:3702 trunc from 0x575d964 to 0x39f8c9c 2024-09-14T07:00:26.276 INFO:teuthology.orchestra.run.smithi007.stdout:3703 writesame 0x6caf7a1 thru 0x6cb9d24 data_size 0x12(0xa584 bytes) 2024-09-14T07:00:26.279 INFO:teuthology.orchestra.run.smithi007.stdout:3705 trunc from 0x6cb9d25 to 0x3d20abf 2024-09-14T07:00:26.285 INFO:teuthology.orchestra.run.smithi007.stdout:3706 read 0x244bc5e thru 0x24565c3 (0xa966 bytes) 2024-09-14T07:00:26.285 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.285 INFO:teuthology.orchestra.run.smithi007.stdout:3711 punch from 0x35ede8a to 0x35f39b2, (0x5b28 bytes) 2024-09-14T07:00:26.285 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.296 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:26.296+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:26.299 INFO:teuthology.orchestra.run.smithi007.stdout:3713 write 0xba4e959 thru 0xba53a06 (0x50ae bytes) 2024-09-14T07:00:26.302 INFO:teuthology.orchestra.run.smithi007.stdout:3714 read 0x3c5ff77 thru 0x3c631d5 (0x325f bytes) 2024-09-14T07:00:26.303 INFO:teuthology.orchestra.run.smithi007.stdout:3715 trunc from 0xba53a07 to 0xbdf7991 2024-09-14T07:00:26.316 INFO:teuthology.orchestra.run.smithi007.stdout:3716 writesame 0xe417bdf thru 0xe41d9e7 data_size 0x13(0x5e09 bytes) 2024-09-14T07:00:26.316 INFO:teuthology.orchestra.run.smithi007.stdout:3717 read 0x5953806 thru 0x5958c2e (0x5429 bytes) 2024-09-14T07:00:26.316 INFO:teuthology.orchestra.run.smithi007.stdout:3718 write 0x4aae520 thru 0x4ab8f28 (0xaa09 bytes) 2024-09-14T07:00:26.319 INFO:teuthology.orchestra.run.smithi007.stdout:3719 write 0xd104b96 thru 0xd10f50e (0xa979 bytes) 2024-09-14T07:00:26.328 INFO:teuthology.orchestra.run.smithi007.stdout:3721 write 0xee13bbf thru 0xee14a88 (0xeca bytes) 2024-09-14T07:00:26.331 INFO:teuthology.orchestra.run.smithi007.stdout:3722 punch from 0x2891f4e to 0x289688e, (0x4940 bytes) 2024-09-14T07:00:26.332 INFO:teuthology.orchestra.run.smithi007.stdout:3723 writesame 0x406c25c thru 0x4079a3b data_size 0x58(0xd7e0 bytes) 2024-09-14T07:00:26.334 INFO:teuthology.orchestra.run.smithi007.stdout:3724 punch from 0x4988387 to 0x49929ab, (0xa624 bytes) 2024-09-14T07:00:26.335 INFO:teuthology.orchestra.run.smithi007.stdout:3725 write 0x735fe3c thru 0x7363d75 (0x3f3a bytes) 2024-09-14T07:00:26.340 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.340 INFO:teuthology.orchestra.run.smithi007.stdout:3728 write 0x5f5beac thru 0x5f69190 (0xd2e5 bytes) 2024-09-14T07:00:26.349 INFO:teuthology.orchestra.run.smithi007.stdout:3729 trunc from 0xee14a89 to 0xd116f20 2024-09-14T07:00:26.359 INFO:teuthology.orchestra.run.smithi007.stdout:3732 read 0x9891116 thru 0x98a0261 (0xf14c bytes) 2024-09-14T07:00:26.359 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.360 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:26.360 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:26.360 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:26.360 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:26.360 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:26.361 INFO:teuthology.orchestra.run.smithi007.stdout:3734 trunc from 0xd116f20 to 0xbe0d3eb 2024-09-14T07:00:26.369 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:26.370 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.370 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.370 INFO:teuthology.orchestra.run.smithi007.stdout:3737 write 0xa5447b7 thru 0xa5515a6 (0xcdf0 bytes) 2024-09-14T07:00:26.377 INFO:teuthology.orchestra.run.smithi007.stdout:3738 trunc from 0xbe0d3eb to 0x86e1736 2024-09-14T07:00:26.386 INFO:teuthology.orchestra.run.smithi007.stdout:3740 write 0x74902f6 thru 0x749dd5b (0xda66 bytes) 2024-09-14T07:00:26.389 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:26.388+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:3741 punch from 0x2233f77 to 0x223c026, (0x80af bytes) 2024-09-14T07:00:26.389 INFO:teuthology.orchestra.run.smithi007.stdout:3743 read 0x7a7734f thru 0x7a790ee (0x1da0 bytes) 2024-09-14T07:00:26.396 INFO:teuthology.orchestra.run.smithi007.stdout:3744 write 0x9f051a7 thru 0x9f13844 (0xe69e bytes) 2024-09-14T07:00:26.399 INFO:teuthology.orchestra.run.smithi007.stdout:3745 read 0x41144cc thru 0x411639b (0x1ed0 bytes) 2024-09-14T07:00:26.399 INFO:teuthology.orchestra.run.smithi007.stdout:3748 write 0x831364b thru 0x831e111 (0xaac7 bytes) 2024-09-14T07:00:26.408 INFO:teuthology.orchestra.run.smithi007.stdout:3750 trunc from 0x9f13845 to 0x4c3f35e 2024-09-14T07:00:26.434 INFO:teuthology.orchestra.run.smithi007.stdout:3751 writesame 0xe0ba20e thru 0xe0bb78e data_size 0x3(0x1581 bytes) 2024-09-14T07:00:26.434 INFO:teuthology.orchestra.run.smithi007.stdout:3752 read 0x426a369 thru 0x42712f2 (0x6f8a bytes) 2024-09-14T07:00:26.434 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.434 INFO:teuthology.orchestra.run.smithi007.stdout:3755 read 0x71a4b thru 0x738b1 (0x1e67 bytes) 2024-09-14T07:00:26.434 INFO:teuthology.orchestra.run.smithi007.stdout:3756 write 0x1a3bce7 thru 0x1a4a970 (0xec8a bytes) 2024-09-14T07:00:26.438 INFO:teuthology.orchestra.run.smithi007.stdout:3757 read 0xb052392 thru 0xb05756e (0x51dd bytes) 2024-09-14T07:00:26.438 INFO:teuthology.orchestra.run.smithi007.stdout:3758 read 0xa1d3c86 thru 0xa1e2493 (0xe80e bytes) 2024-09-14T07:00:26.438 INFO:teuthology.orchestra.run.smithi007.stdout:3759 read 0x6b8943c thru 0x6b975ee (0xe1b3 bytes) 2024-09-14T07:00:26.443 INFO:teuthology.orchestra.run.smithi007.stdout:3760 trunc from 0xe0bb78f to 0x9b97e91 2024-09-14T07:00:26.448 INFO:teuthology.orchestra.run.smithi007.stdout:3761 trunc from 0x9b97e91 to 0x989736e 2024-09-14T07:00:26.453 INFO:teuthology.orchestra.run.smithi007.stdout:3762 read 0x8c93199 thru 0x8c9e6d7 (0xb53f bytes) 2024-09-14T07:00:26.453 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.453 INFO:teuthology.orchestra.run.smithi007.stdout:3764 writesame 0x669a23 thru 0x67354c data_size 0x1(0x9b2a bytes) 2024-09-14T07:00:26.484 INFO:teuthology.orchestra.run.smithi007.stdout:3765 writesame 0xe143d90 thru 0xe151338 data_size 0x1(0xd5a9 bytes) 2024-09-14T07:00:26.484 INFO:teuthology.orchestra.run.smithi007.stdout:3766 write 0x8f97a65 thru 0x8fa321b (0xb7b7 bytes) 2024-09-14T07:00:26.487 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.488 INFO:teuthology.orchestra.run.smithi007.stdout:3769 read 0x10ce602 thru 0x10d0a9b (0x249a bytes) 2024-09-14T07:00:26.490 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:26.489+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:26.491 INFO:teuthology.orchestra.run.smithi007.stdout:3770 trunc from 0xe151339 to 0xae71d99 2024-09-14T07:00:26.498 INFO:teuthology.orchestra.run.smithi007.stdout:3772 trunc from 0xae71d99 to 0x93a35d8 2024-09-14T07:00:26.503 INFO:teuthology.orchestra.run.smithi007.stdout:3773 read 0x7ab0fc6 thru 0x7ab633b (0x5376 bytes) 2024-09-14T07:00:26.503 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.504 INFO:teuthology.orchestra.run.smithi007.stdout:3775 punch from 0x7911d35 to 0x791ccc2, (0xaf8d bytes) 2024-09-14T07:00:26.504 INFO:teuthology.orchestra.run.smithi007.stdout:3776 read 0x867d5c1 thru 0x867ed7b (0x17bb bytes) 2024-09-14T07:00:26.515 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.2. 2024-09-14T07:00:26.519 INFO:teuthology.orchestra.run.smithi007.stdout:3778 write 0xbaf65d0 thru 0xbafc87a (0x62ab bytes) 2024-09-14T07:00:26.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:00:26.524 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:26.524 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:26.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:26.524 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:26.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.525 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131358 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.525 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:26.525 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022871438999999, 2024-09-14T07:00:26.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030805583, 2024-09-14T07:00:26.525 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.525 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.526 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.526 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.526 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.526 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.526 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131358 2024-09-14T07:00:26.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.527 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:26.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:26.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:26.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:00:26.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498000+0000", 2024-09-14T07:00:26.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:00:26.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498004+0000", 2024-09-14T07:00:26.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2569999999999999e-06 2024-09-14T07:00:26.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498007+0000", 2024-09-14T07:00:26.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.507e-06 2024-09-14T07:00:26.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499958+0000", 2024-09-14T07:00:26.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001950989 2024-09-14T07:00:26.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499962+0000", 2024-09-14T07:00:26.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-14T07:00:26.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528792+0000", 2024-09-14T07:00:26.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288306389999999 2024-09-14T07:00:26.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528804+0000", 2024-09-14T07:00:26.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2096000000000001e-05 2024-09-14T07:00:26.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.535 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.536 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131359 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.536 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:26.536 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022852028999999, 2024-09-14T07:00:26.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308363920000001, 2024-09-14T07:00:26.537 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.537 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.537 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.537 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.537 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.537 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.538 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131359 2024-09-14T07:00:26.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.538 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:26.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:26.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498019+0000", 2024-09-14T07:00:26.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:00:26.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498020+0000", 2024-09-14T07:00:26.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4529999999999999e-06 2024-09-14T07:00:26.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498023+0000", 2024-09-14T07:00:26.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3450000000000002e-06 2024-09-14T07:00:26.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498026+0000", 2024-09-14T07:00:26.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.661e-06 2024-09-14T07:00:26.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499995+0000", 2024-09-14T07:00:26.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00196926 2024-09-14T07:00:26.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500002+0000", 2024-09-14T07:00:26.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6930000000000001e-06 2024-09-14T07:00:26.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528842+0000", 2024-09-14T07:00:26.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288401300000001 2024-09-14T07:00:26.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528855+0000", 2024-09-14T07:00:26.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2595999999999999e-05 2024-09-14T07:00:26.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131360 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022841026, 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030841943, 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.545 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131360 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498030+0000", 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498031+0000", 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1119999999999999e-06 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498034+0000", 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498036+0000", 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.599e-06 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500030+0000", 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001993613 2024-09-14T07:00:26.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500034+0000", 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1060000000000002e-06 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528867+0000", 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288327429999999 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528871+0000", 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.673e-06 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131361 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022820281, 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308644520000001, 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.550 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131361 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:26.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498052+0000", 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.24e-06 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498055+0000", 2024-09-14T07:00:26.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1609999999999999e-06 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498058+0000", 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.05e-06 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500066+0000", 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020081169999999998 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500075+0000", 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0559999999999994e-06 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528903+0000", 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288276810000001 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528915+0000", 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1938e-05 2024-09-14T07:00:26.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131362 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.0228015, 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030860661, 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.555 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131362 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498075+0000", 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5639999999999996e-06 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498079+0000", 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498081+0000", 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8660000000000002e-06 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500101+0000", 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020191219999999999 2024-09-14T07:00:26.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500104+0000", 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6890000000000002e-06 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528925+0000", 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028820971 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528930+0000", 2024-09-14T07:00:26.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3390000000000003e-06 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131363 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022785252, 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030874107, 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.560 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.561 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.561 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131363 2024-09-14T07:00:26.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.561 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498086+0000", 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1009999999999999e-06 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498090+0000", 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5609999999999999e-06 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498093+0000", 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7360000000000001e-06 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500136+0000", 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020436030000000002 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500145+0000", 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9530000000000005e-06 2024-09-14T07:00:26.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528948+0000", 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028803103 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528959+0000", 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0808999999999999e-05 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.617 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131364 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022768871, 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308734129999999, 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.618 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131364 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498102+0000", 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498103+0000", 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:26.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498107+0000", 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4089999999999999e-06 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498110+0000", 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7760000000000002e-06 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500177+0000", 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020678200000000002 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500183+0000", 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4979999999999997e-06 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528970+0000", 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287867049999999 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528975+0000", 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1959999999999999e-06 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131365 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022746379999999, 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886024, 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.623 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131365 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:26.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:26.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498125+0000", 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.17e-06 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498129+0000", 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498131+0000", 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.666e-06 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500210+0000", 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020789559999999999 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500215+0000", 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:26.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528993+0000", 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028777174 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529010+0000", 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7419999999999999e-05 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131366 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022728082, 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308831409999999, 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131366 2024-09-14T07:00:26.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498143+0000", 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498144+0000", 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.474e-06 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498147+0000", 2024-09-14T07:00:26.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.371e-06 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498150+0000", 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6410000000000002e-06 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500239+0000", 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002088842 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500243+0000", 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6689999999999999e-06 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529020+0000", 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287776420000001 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529025+0000", 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2440000000000001e-06 2024-09-14T07:00:26.633 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131367 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022717022, 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030895135, 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.634 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131367 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498154+0000", 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2100000000000001e-07 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498155+0000", 2024-09-14T07:00:26.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1310000000000001e-06 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498157+0000", 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7870000000000002e-06 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498160+0000", 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9009999999999998e-06 2024-09-14T07:00:26.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500264+0000", 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021031489999999999 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500268+0000", 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0929999999999996e-06 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529043+0000", 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287752139999999 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529048+0000", 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6389999999999997e-06 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131368 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.02269289, 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308865389999999, 2024-09-14T07:00:26.639 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131368 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498178+0000", 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8200000000000001e-07 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498179+0000", 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6929999999999999e-06 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498183+0000", 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9439999999999998e-06 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498188+0000", 2024-09-14T07:00:26.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9409999999999998e-06 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500287+0000", 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020982269999999998 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500290+0000", 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6349999999999999e-06 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529059+0000", 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028768648 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529064+0000", 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.169e-06 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.644 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131369 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022677698000001, 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886017, 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131369 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.28e-07 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498194+0000", 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1459999999999999e-06 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498197+0000", 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0620000000000001e-06 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498200+0000", 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6000000000000001e-06 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500310+0000", 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021101840000000002 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500314+0000", 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.126e-06 2024-09-14T07:00:26.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529074+0000", 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.02875961 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529079+0000", 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0610000000000003e-06 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131370 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022661905, 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308930119999999, 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131370 2024-09-14T07:00:26.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498209+0000", 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498210+0000", 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.187e-06 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498213+0000", 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.151e-06 2024-09-14T07:00:26.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498216+0000", 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8289999999999999e-06 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500340+0000", 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021244229999999998 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500349+0000", 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529096+0000", 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287473309999999 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529101+0000", 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131371 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022643524999999, 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308901880000001, 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131371 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.655 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498229+0000", 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1110000000000002e-06 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498234+0000", 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498239+0000", 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.651e-06 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500382+0000", 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021424780000000002 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500389+0000", 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529112+0000", 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287229410000001 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529117+0000", 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9520000000000002e-06 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.659 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131372 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022622409, 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030884366, 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131372 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8599999999999999e-07 2024-09-14T07:00:26.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498251+0000", 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498257+0000", 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6370000000000002e-06 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498262+0000", 2024-09-14T07:00:26.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500418+0000", 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021561800000000002 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500423+0000", 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1520000000000002e-06 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529127+0000", 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287049150000001 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529132+0000", 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.865e-06 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.664 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131373 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022589346, 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308673479999999, 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131373 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.665 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:26.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498283+0000", 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498290+0000", 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498295+0000", 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3759999999999999e-06 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500441+0000", 2024-09-14T07:00:26.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021462849999999999 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500445+0000", 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8310000000000003e-06 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529142+0000", 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028697363 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529148+0000", 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131374 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022555730000001, 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308477090000001, 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.670 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131374 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498322+0000", 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5760000000000002e-06 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498328+0000", 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4609999999999996e-06 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498334+0000", 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7130000000000002e-06 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500464+0000", 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002130617 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500468+0000", 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7340000000000002e-06 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529157+0000", 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286893669999999 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529162+0000", 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131375 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022515593, 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308296320000001, 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.675 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131375 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498358+0000", 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.317e-06 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498364+0000", 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5930000000000002e-06 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498369+0000", 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3630000000000002e-06 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500488+0000", 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021187699999999999 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500492+0000", 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529179+0000", 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286873590000001 2024-09-14T07:00:26.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529184+0000", 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131376 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022488054, 2024-09-14T07:00:26.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030817713, 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131376 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498383+0000", 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498385+0000", 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.091e-06 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498391+0000", 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1070000000000001e-06 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498396+0000", 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4670000000000003e-06 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500514+0000", 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021177449999999999 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500518+0000", 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8770000000000003e-06 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529195+0000", 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028676938 2024-09-14T07:00:26.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529200+0000", 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131377 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 10.022448529, 2024-09-14T07:00:26.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308008749999999, 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131377 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498425+0000", 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5620000000000002e-06 2024-09-14T07:00:26.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498431+0000", 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2539999999999998e-06 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498436+0000", 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.31e-06 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500539+0000", 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021025459999999998 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500543+0000", 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.597e-06 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529218+0000", 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286753930000001 2024-09-14T07:00:26.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529223+0000", 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.758e-06 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.690 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:26.691 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:26.691 INFO:teuthology.orchestra.run.smithi007.stdout:3782 read 0x8f6646f thru 0x8f6f847 (0x93d9 bytes) 2024-09-14T07:00:26.693 INFO:teuthology.orchestra.run.smithi007.stdout:3783 trunc from 0xbafc87b to 0x2a82131 2024-09-14T07:00:26.693 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:26.694 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-14T07:00:26.694 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:26.691+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:26.792 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:26.792+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:26.797 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:26.797 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:26.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.797 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:138070 3.4 3.b0dcd84 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-14T07:00:26.797 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:26.535995+0000", 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.26143155800000001, 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.26145143500000001, 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 138070 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.535995+0000", 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.535995+0000", 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.535997+0000", 2024-09-14T07:00:26.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6279999999999999e-06 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.536003+0000", 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0680000000000002e-06 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.536019+0000", 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5917e-05 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.536028+0000", 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6940000000000001e-06 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.536087+0000", 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000002e-05 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:26.536098+0000", 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0548e-05 2024-09-14T07:00:26.801 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.802 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:26.802 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.802 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:26.802 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-14T07:00:26.802 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 1 2024-09-14T07:00:26.802 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:26.806 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:26.893 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:26.893+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:26.947 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:26.948 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:26.948 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:26.948 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:26.948 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:26.956 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:26.998 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:26.998+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:27.061 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:27.061 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:27.061 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:27.061 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:27.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.061 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:27.061 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.894282615999998, 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:00:27.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.242939510999999, 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.066 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:00:27.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.242924042999999, 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.071 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.072 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.072 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.072 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:00:27.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.072 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:27.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.073 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.073 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:27.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.073 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:00:27.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.076 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.242912742000001, 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.077 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:00:27.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236169269000001, 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.081 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:00:27.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236156664999999, 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.086 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:00:27.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236137149000001, 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:00:27.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236125108, 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:27.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236104806, 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:00:27.100 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:00:27.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.104 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236092490000001, 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.105 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:00:27.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.109 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236081049999999, 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.113 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236070551000001, 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.114 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:27.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:00:27.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236060358, 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.118 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:27.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:00:27.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.122 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236044810999999, 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:00:27.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:00:27.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:27.127 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.236015203000001, 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:00:27.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.131 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.235993553, 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.132 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:00:27.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:00:27.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.235971314, 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.136 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.140 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.230374949000002, 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.143 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.144 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 19.230364896000001, 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.145 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:118426 3.b 3:d50c2793:::rbd_data.114f74d20483.000000000000001c:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 14.558983093, 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.95135023100000005, 2024-09-14T07:00:27.149 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 118426 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500745+0000", 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:27.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500748+0000", 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1329999999999999e-06 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500759+0000", 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0475000000000001e-05 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500765+0000", 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500824+0000", 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500830+0000", 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5889999999999999e-06 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450787+0000", 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.94995644099999998 2024-09-14T07:00:27.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450901+0000", 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114032 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451052+0000", 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015113 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:27.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451746+0000", 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069442799999999997 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452006+0000", 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025975 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452059+0000", 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3090000000000002e-05 2024-09-14T07:00:27.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452095+0000", 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5395999999999998e-05 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:27.156 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:27.158 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:27.158 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:27.157+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:27.258 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:27.258+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:27.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:27.302 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:27.302 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:27.302 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:27.302 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:27.311 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:27.394 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:27.394+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:27.399 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:27.399 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:27.416 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:27.416 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:27.432 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:27.448 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:27.454 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:27.454 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:27.454 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:27.454 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:27.454 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:27.460 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:27.459+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:27.463 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:27.466 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:27.466 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:27.483 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:27.483 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:27.500 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:27.500 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T07:00:27.523 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:27.523 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:27.576 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:27.575+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:27.607 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:27.608 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:27.616 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:27.663 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:27.662+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:27.718 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:27.718 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:27.727 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:27.770 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:27.770+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:27.831 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:27.831 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:27.840 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:27.881 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:27.880+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:27.965 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:27.964+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:27.983 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:27.983 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:27.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:28.066 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:28.065+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:28.105 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:28.105 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:28.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:28.167 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:28.166+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:28.220 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:28.220 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:28.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:28.268 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:28.267+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:28.369 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:28.369+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:28.376 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:28.376 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:28.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:28.470 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:28.470+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:28.491 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:28.491 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:28.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:28.571 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:28.570+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:28.612 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:28.612 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:28.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:28.672 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:28.671+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:28.698 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:28.699 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:28.701 INFO:teuthology.orchestra.run.smithi007.stdout:3788 trunc from 0x2a82131 to 0x5263f14 2024-09-14T07:00:28.708 INFO:teuthology.orchestra.run.smithi007.stdout:3790 writesame 0x43f9f53 thru 0x44075f0 data_size 0x3(0xd69e bytes) 2024-09-14T07:00:28.719 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:28.727 INFO:teuthology.orchestra.run.smithi007.stdout:3792 trunc from 0x5263f14 to 0xc13152d 2024-09-14T07:00:28.743 INFO:teuthology.orchestra.run.smithi007.stdout:3794 trunc from 0xc13152d to 0x3bd8f1c 2024-09-14T07:00:28.760 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:28.760 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:28.769 INFO:teuthology.orchestra.run.smithi007.stdout:3795 writesame 0xc4cb26c thru 0xc4cbb03 data_size 0xb(0x898 bytes) 2024-09-14T07:00:28.769 INFO:teuthology.orchestra.run.smithi007.stdout:3796 writesame 0x90dfe57 thru 0x90e7987 data_size 0x2f(0x7b31 bytes) 2024-09-14T07:00:28.772 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:28.772+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:28.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:28.776 INFO:teuthology.orchestra.run.smithi007.stdout:3797 punch from 0x1fb55d9 to 0x1fbe59c, (0x8fc3 bytes) 2024-09-14T07:00:28.776 INFO:teuthology.orchestra.run.smithi007.stdout:3799 write 0xb55c45b thru 0xb562d3d (0x68e3 bytes) 2024-09-14T07:00:28.790 INFO:teuthology.orchestra.run.smithi007.stdout:3803 write 0xd18f5f6 thru 0xd198dba (0x97c5 bytes) 2024-09-14T07:00:28.794 INFO:teuthology.orchestra.run.smithi007.stdout:3804 punch from 0x57cc605 to 0x57d6885, (0xa280 bytes) 2024-09-14T07:00:28.794 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:28.794 INFO:teuthology.orchestra.run.smithi007.stdout:3806 writesame 0x20733df thru 0x2082a3f data_size 0x1(0xf661 bytes) 2024-09-14T07:00:28.829 INFO:teuthology.orchestra.run.smithi007.stdout:3807 punch from 0xbcad08a to 0xbcae827, (0x179d bytes) 2024-09-14T07:00:28.829 INFO:teuthology.orchestra.run.smithi007.stdout:3808 write 0xb4eae4a thru 0xb4f4090 (0x9247 bytes) 2024-09-14T07:00:28.832 INFO:teuthology.orchestra.run.smithi007.stdout:3809 write 0x1a627fb thru 0x1a6902c (0x6832 bytes) 2024-09-14T07:00:28.836 INFO:teuthology.orchestra.run.smithi007.stdout:3810 read 0x2ad3474 thru 0x2ad3762 (0x2ef bytes) 2024-09-14T07:00:28.836 INFO:teuthology.orchestra.run.smithi007.stdout:3811 read 0x4b708e9 thru 0x4b76b33 (0x624b bytes) 2024-09-14T07:00:28.836 INFO:teuthology.orchestra.run.smithi007.stdout:3816 write 0x4833fd6 thru 0x4836564 (0x258f bytes) 2024-09-14T07:00:28.850 INFO:teuthology.orchestra.run.smithi007.stdout:3817 trunc from 0xd198dbb to 0x45d345e 2024-09-14T07:00:28.862 INFO:teuthology.orchestra.run.smithi007.stdout:3819 read 0x13e24b4 thru 0x13ef999 (0xd4e6 bytes) 2024-09-14T07:00:28.862 INFO:teuthology.orchestra.run.smithi007.stdout:3820 punch from 0xb852ea to 0xb93697, (0xe3ad bytes) 2024-09-14T07:00:28.871 INFO:teuthology.orchestra.run.smithi007.stdout:3821 writesame 0x73615d1 thru 0x7364f30 data_size 0x36(0x3960 bytes) 2024-09-14T07:00:28.873 INFO:teuthology.orchestra.run.smithi007.stdout:3822 trunc from 0x7364f31 to 0x8e7d0f2 2024-09-14T07:00:28.873 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:28.872+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:28.878 INFO:teuthology.orchestra.run.smithi007.stdout:3823 write 0x21da74a thru 0x21e2aac (0x8363 bytes) 2024-09-14T07:00:28.882 INFO:teuthology.orchestra.run.smithi007.stdout:3824 writesame 0x121e6a0 thru 0x12229b6 data_size 0x19(0x4317 bytes) 2024-09-14T07:00:28.891 INFO:teuthology.orchestra.run.smithi007.stdout:3827 writesame 0x7a32b63 thru 0x7a3a483 data_size 0x1(0x7921 bytes) 2024-09-14T07:00:28.906 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:28.907 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:28.915 INFO:teuthology.orchestra.run.smithi007.stdout:3829 trunc from 0x8e7d0f2 to 0x26fe9eb 2024-09-14T07:00:28.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:28.926 INFO:teuthology.orchestra.run.smithi007.stdout:3831 read 0xd3dafd thru 0xd48cb8 (0xb1bc bytes) 2024-09-14T07:00:28.941 INFO:teuthology.orchestra.run.smithi007.stdout:3832 writesame 0x9b7dfc3 thru 0x9b822e5 data_size 0x33(0x4323 bytes) 2024-09-14T07:00:28.943 INFO:teuthology.orchestra.run.smithi007.stdout:3834 trunc from 0x9b822e6 to 0xbe3d803 2024-09-14T07:00:28.949 INFO:teuthology.orchestra.run.smithi007.stdout:3836 write 0x26a206b thru 0x26ae99f (0xc935 bytes) 2024-09-14T07:00:28.953 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:28.961 INFO:teuthology.orchestra.run.smithi007.stdout:3839 write 0xd645bf9 thru 0xd64b858 (0x5c60 bytes) 2024-09-14T07:00:28.966 INFO:teuthology.orchestra.run.smithi007.stdout:3841 trunc from 0xd64b859 to 0xc25bb56 2024-09-14T07:00:28.974 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:28.973+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:28.976 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:28.976 INFO:teuthology.orchestra.run.smithi007.stdout:3843 write 0x7d89ce thru 0x7de52d (0x5b60 bytes) 2024-09-14T07:00:28.980 INFO:teuthology.orchestra.run.smithi007.stdout:3844 write 0x41470c9 thru 0x4147207 (0x13f bytes) 2024-09-14T07:00:28.983 INFO:teuthology.orchestra.run.smithi007.stdout:3845 writesame 0xba499eb thru 0xba59166 data_size 0x2f(0xf77c bytes) 2024-09-14T07:00:28.989 INFO:teuthology.orchestra.run.smithi007.stdout:3847 punch from 0x91a0e29 to 0x91af07d, (0xe254 bytes) 2024-09-14T07:00:28.989 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:28.995 INFO:teuthology.orchestra.run.smithi007.stdout:3850 trunc from 0xc25bb56 to 0x6cae366 2024-09-14T07:00:29.007 INFO:teuthology.orchestra.run.smithi007.stdout:3851 punch from 0x5c588b1 to 0x5c67292, (0xe9e1 bytes) 2024-09-14T07:00:29.007 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:29.017 INFO:teuthology.orchestra.run.smithi007.stdout:3854 write 0xa28d276 thru 0xa299585 (0xc310 bytes) 2024-09-14T07:00:29.024 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:29.024 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:29.025 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:29.025 INFO:teuthology.orchestra.run.smithi007.stdout:3860 read 0x88f159f thru 0x88fbc20 (0xa682 bytes) 2024-09-14T07:00:29.025 INFO:teuthology.orchestra.run.smithi007.stdout:3861 trunc from 0xa299586 to 0xd98bbc7 2024-09-14T07:00:29.031 INFO:teuthology.orchestra.run.smithi007.stdout:3863 read 0x894d48b thru 0x8951240 (0x3db6 bytes) 2024-09-14T07:00:29.031 INFO:teuthology.orchestra.run.smithi007.stdout:3864 read 0x5db5c69 thru 0x5dc2696 (0xca2e bytes) 2024-09-14T07:00:29.031 INFO:teuthology.orchestra.run.smithi007.stdout:3865 read 0xd0def1 thru 0xd15674 (0x7784 bytes) 2024-09-14T07:00:29.031 INFO:teuthology.orchestra.run.smithi007.stdout:3866 read 0xc8793aa thru 0xc88734c (0xdfa3 bytes) 2024-09-14T07:00:29.032 INFO:teuthology.orchestra.run.smithi007.stdout:3868 write 0xc3d8f65 thru 0xc3e1f45 (0x8fe1 bytes) 2024-09-14T07:00:29.038 INFO:teuthology.orchestra.run.smithi007.stdout:3869 punch from 0xc5148cb to 0xc51e172, (0x98a7 bytes) 2024-09-14T07:00:29.038 INFO:teuthology.orchestra.run.smithi007.stdout:3873 writesame 0x89e8d4e thru 0x89efc55 data_size 0x58(0x6f08 bytes) 2024-09-14T07:00:29.042 INFO:teuthology.orchestra.run.smithi007.stdout:3874 write 0xc4504ab thru 0xc457f29 (0x7a7f bytes) 2024-09-14T07:00:29.045 INFO:teuthology.orchestra.run.smithi007.stdout:3875 writesame 0x600001 thru 0x60b801 data_size 0x1(0xb801 bytes) 2024-09-14T07:00:29.066 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:29.066 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:29.074 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:29.074+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:29.075 INFO:teuthology.orchestra.run.smithi007.stdout:3876 trunc from 0xd98bbc7 to 0xab0855a 2024-09-14T07:00:29.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:29.085 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:29.085 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:29.085 INFO:teuthology.orchestra.run.smithi007.stdout:3879 write 0x8048ef0 thru 0x804fb18 (0x6c29 bytes) 2024-09-14T07:00:29.088 INFO:teuthology.orchestra.run.smithi007.stdout:3882 punch from 0x2ffb5cd to 0x300a48c, (0xeebf bytes) 2024-09-14T07:00:29.088 INFO:teuthology.orchestra.run.smithi007.stdout:3883 read 0x5a880f4 thru 0x5a8e43d (0x634a bytes) 2024-09-14T07:00:29.093 INFO:teuthology.orchestra.run.smithi007.stdout:3884 trunc from 0xab0855a to 0x699dd7f 2024-09-14T07:00:29.104 INFO:teuthology.orchestra.run.smithi007.stdout:3885 read 0x447a90e thru 0x447bb15 (0x1208 bytes) 2024-09-14T07:00:29.104 INFO:teuthology.orchestra.run.smithi007.stdout:3887 read 0xbeafb2 thru 0xbf4313 (0x9362 bytes) 2024-09-14T07:00:29.104 INFO:teuthology.orchestra.run.smithi007.stdout:3890 write 0x9f343b thru 0x9faab4 (0x767a bytes) 2024-09-14T07:00:29.107 INFO:teuthology.orchestra.run.smithi007.stdout:3891 punch from 0x4563d7a to 0x456e009, (0xa28f bytes) 2024-09-14T07:00:29.119 INFO:teuthology.orchestra.run.smithi007.stdout:3893 write 0xb99123b thru 0xb9a08a9 (0xf66f bytes) 2024-09-14T07:00:29.123 INFO:teuthology.orchestra.run.smithi007.stdout:3895 write 0x4eb2ac4 thru 0x4eb5cab (0x31e8 bytes) 2024-09-14T07:00:29.137 INFO:teuthology.orchestra.run.smithi007.stdout:3896 trunc from 0xb9a08aa to 0x2d6f8ef 2024-09-14T07:00:29.148 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:29.148 INFO:teuthology.orchestra.run.smithi007.stdout:3898 write 0x2d045cb thru 0x2d0d23a (0x8c70 bytes) 2024-09-14T07:00:29.161 INFO:teuthology.orchestra.run.smithi007.stdout:3899 write 0x6f5695c thru 0x6f59933 (0x2fd8 bytes) 2024-09-14T07:00:29.165 INFO:teuthology.orchestra.run.smithi007.stdout:3900 writesame 0xa7242b thru 0xa782a0 data_size 0x6b(0x5e76 bytes) 2024-09-14T07:00:29.175 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:29.174+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:29.182 INFO:teuthology.orchestra.run.smithi007.stdout:3901 write 0xc70777d thru 0xc714e02 (0xd686 bytes) 2024-09-14T07:00:29.185 INFO:teuthology.orchestra.run.smithi007.stdout:3902 write 0x5951122 thru 0x595547d (0x435c bytes) 2024-09-14T07:00:29.189 INFO:teuthology.orchestra.run.smithi007.stdout:3903 read 0x3cd735 thru 0x3d70e9 (0x99b5 bytes) 2024-09-14T07:00:29.190 INFO:teuthology.orchestra.run.smithi007.stdout:3904 write 0x467098a thru 0x467453d (0x3bb4 bytes) 2024-09-14T07:00:29.206 INFO:teuthology.orchestra.run.smithi007.stdout:3907 trunc from 0xc714e03 to 0x140739f 2024-09-14T07:00:29.220 INFO:teuthology.orchestra.run.smithi007.stdout:3908 punch from 0x7db84e to 0x7e0e8f, (0x5641 bytes) 2024-09-14T07:00:29.230 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:29.230 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:29.232 INFO:teuthology.orchestra.run.smithi007.stdout:3909 writesame 0x644ffdb thru 0x6450a5c data_size 0x5(0xa82 bytes) 2024-09-14T07:00:29.243 INFO:teuthology.orchestra.run.smithi007.stdout:3910 write 0xae427f0 thru 0xae49f61 (0x7772 bytes) 2024-09-14T07:00:29.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:29.247 INFO:teuthology.orchestra.run.smithi007.stdout:3911 punch from 0x7be7524 to 0x7befe98, (0x8974 bytes) 2024-09-14T07:00:29.247 INFO:teuthology.orchestra.run.smithi007.stdout:3913 punch from 0x907c1c3 to 0x9080d05, (0x4b42 bytes) 2024-09-14T07:00:29.247 INFO:teuthology.orchestra.run.smithi007.stdout:3916 punch from 0x2eaff59 to 0x2eba6ed, (0xa794 bytes) 2024-09-14T07:00:29.247 INFO:teuthology.orchestra.run.smithi007.stdout:3917 punch from 0x30532a1 to 0x3059d67, (0x6ac6 bytes) 2024-09-14T07:00:29.253 INFO:teuthology.orchestra.run.smithi007.stdout:3918 trunc from 0xae49f62 to 0x53e31e3 2024-09-14T07:00:29.265 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:29.265 INFO:teuthology.orchestra.run.smithi007.stdout:3921 punch from 0x2c40cee to 0x2c44bbe, (0x3ed0 bytes) 2024-09-14T07:00:29.266 INFO:teuthology.orchestra.run.smithi007.stdout:3922 read 0x1620c7e thru 0x16249a6 (0x3d29 bytes) 2024-09-14T07:00:29.276 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:29.275+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:29.287 INFO:teuthology.orchestra.run.smithi007.stdout:3923 write 0xcbb3333 thru 0xcbb6f33 (0x3c01 bytes) 2024-09-14T07:00:29.376 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:29.376+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:29.377 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:29.377 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:29.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:29.477 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:29.476+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:29.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:00:29.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:29.551 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:29.551 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:29.562 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:29.578 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:29.577+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:29.678 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:29.678+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:29.693 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:29.693 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:29.723 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:29.723 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:29.735 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:29.745 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:29.756 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:29.756 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:29.779 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:29.778+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:29.780 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:29.780 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:29.815 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:29.833 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:29.833 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:29.850 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:29.850 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:29.850 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:29.851 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:29.852 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:29.859 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:29.869 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:29.869 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:29.903 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:29.902+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:29.962 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:29.962 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:29.963 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:29.963 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:29.971 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:29.980 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:29.980+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:30.082 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:30.081+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:30.118 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:30.118 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:30.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:30.118 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:30.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.118 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131358 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.118 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:30.118 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.61835226, 2024-09-14T07:00:30.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030805583, 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131358 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:30.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498000+0000", 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498004+0000", 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2569999999999999e-06 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498007+0000", 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.507e-06 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499958+0000", 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001950989 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499962+0000", 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528792+0000", 2024-09-14T07:00:30.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288306389999999 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528804+0000", 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2096000000000001e-05 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131359 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:30.123 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.61833285, 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308363920000001, 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131359 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498019+0000", 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498020+0000", 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4529999999999999e-06 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498023+0000", 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3450000000000002e-06 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498026+0000", 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.661e-06 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499995+0000", 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00196926 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500002+0000", 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6930000000000001e-06 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528842+0000", 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288401300000001 2024-09-14T07:00:30.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528855+0000", 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2595999999999999e-05 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131360 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618321847000001, 2024-09-14T07:00:30.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030841943, 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131360 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498030+0000", 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498031+0000", 2024-09-14T07:00:30.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1119999999999999e-06 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498034+0000", 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498036+0000", 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.599e-06 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500030+0000", 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001993613 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500034+0000", 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1060000000000002e-06 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528867+0000", 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288327429999999 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528871+0000", 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.673e-06 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131361 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618301102, 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308644520000001, 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.133 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131361 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498052+0000", 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.24e-06 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498055+0000", 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1609999999999999e-06 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498058+0000", 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.05e-06 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500066+0000", 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020081169999999998 2024-09-14T07:00:30.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500075+0000", 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0559999999999994e-06 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528903+0000", 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288276810000001 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528915+0000", 2024-09-14T07:00:30.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1938e-05 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131362 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618282321000001, 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030860661, 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131362 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498075+0000", 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5639999999999996e-06 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498079+0000", 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498081+0000", 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8660000000000002e-06 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500101+0000", 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020191219999999999 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.141 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500104+0000", 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6890000000000002e-06 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528925+0000", 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028820971 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528930+0000", 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3390000000000003e-06 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.142 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131363 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618266073000001, 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030874107, 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.143 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131363 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.144 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498086+0000", 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1009999999999999e-06 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498090+0000", 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5609999999999999e-06 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.145 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498093+0000", 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7360000000000001e-06 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500136+0000", 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020436030000000002 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500145+0000", 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9530000000000005e-06 2024-09-14T07:00:30.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528948+0000", 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028803103 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528959+0000", 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0808999999999999e-05 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131364 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618249691999999, 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308734129999999, 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131364 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.148 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498102+0000", 2024-09-14T07:00:30.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498103+0000", 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498107+0000", 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4089999999999999e-06 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498110+0000", 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7760000000000002e-06 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500177+0000", 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020678200000000002 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500183+0000", 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4979999999999997e-06 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528970+0000", 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287867049999999 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528975+0000", 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1959999999999999e-06 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131365 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.152 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618227201, 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886024, 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131365 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.153 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498125+0000", 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.17e-06 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498129+0000", 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498131+0000", 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.666e-06 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500210+0000", 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020789559999999999 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500215+0000", 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528993+0000", 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028777174 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529010+0000", 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7419999999999999e-05 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131366 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618208902999999, 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308831409999999, 2024-09-14T07:00:30.157 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131366 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498143+0000", 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498144+0000", 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.474e-06 2024-09-14T07:00:30.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498147+0000", 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.371e-06 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498150+0000", 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6410000000000002e-06 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500239+0000", 2024-09-14T07:00:30.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002088842 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500243+0000", 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6689999999999999e-06 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529020+0000", 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287776420000001 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.161 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529025+0000", 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2440000000000001e-06 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131367 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618197843000001, 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030895135, 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.162 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131367 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:30.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498154+0000", 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2100000000000001e-07 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498155+0000", 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1310000000000001e-06 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498157+0000", 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7870000000000002e-06 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498160+0000", 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9009999999999998e-06 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500264+0000", 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021031489999999999 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500268+0000", 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0929999999999996e-06 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529043+0000", 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287752139999999 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529048+0000", 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6389999999999997e-06 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131368 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618173711000001, 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308865389999999, 2024-09-14T07:00:30.167 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131368 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498178+0000", 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8200000000000001e-07 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498179+0000", 2024-09-14T07:00:30.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6929999999999999e-06 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498183+0000", 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9439999999999998e-06 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498188+0000", 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9409999999999998e-06 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500287+0000", 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020982269999999998 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500290+0000", 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6349999999999999e-06 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529059+0000", 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028768648 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529064+0000", 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.169e-06 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131369 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618158519, 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886017, 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.172 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131369 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.28e-07 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498194+0000", 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1459999999999999e-06 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498197+0000", 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0620000000000001e-06 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498200+0000", 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6000000000000001e-06 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500310+0000", 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021101840000000002 2024-09-14T07:00:30.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500314+0000", 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.126e-06 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529074+0000", 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.02875961 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529079+0000", 2024-09-14T07:00:30.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0610000000000003e-06 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131370 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618142726, 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308930119999999, 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.177 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131370 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498209+0000", 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498210+0000", 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.187e-06 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498213+0000", 2024-09-14T07:00:30.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.151e-06 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498216+0000", 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8289999999999999e-06 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500340+0000", 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021244229999999998 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500349+0000", 2024-09-14T07:00:30.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529096+0000", 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287473309999999 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529101+0000", 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131371 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618124346, 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308901880000001, 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131371 2024-09-14T07:00:30.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498229+0000", 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1110000000000002e-06 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498234+0000", 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498239+0000", 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.651e-06 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500382+0000", 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021424780000000002 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500389+0000", 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529112+0000", 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287229410000001 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529117+0000", 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9520000000000002e-06 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131372 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:30.186 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618103229999999, 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030884366, 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131372 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:30.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8599999999999999e-07 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498251+0000", 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498257+0000", 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6370000000000002e-06 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498262+0000", 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500418+0000", 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021561800000000002 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500423+0000", 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1520000000000002e-06 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529127+0000", 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287049150000001 2024-09-14T07:00:30.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529132+0000", 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.865e-06 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131373 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618070166999999, 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308673479999999, 2024-09-14T07:00:30.191 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131373 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498283+0000", 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:30.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498290+0000", 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498295+0000", 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3759999999999999e-06 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500441+0000", 2024-09-14T07:00:30.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021462849999999999 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500445+0000", 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8310000000000003e-06 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529142+0000", 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028697363 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529148+0000", 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131374 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.618036550999999, 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308477090000001, 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.196 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131374 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:30.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498322+0000", 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5760000000000002e-06 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498328+0000", 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4609999999999996e-06 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498334+0000", 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7130000000000002e-06 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500464+0000", 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002130617 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500468+0000", 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7340000000000002e-06 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529157+0000", 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286893669999999 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529162+0000", 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.200 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131375 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.617996414, 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308296320000001, 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.201 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131375 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498358+0000", 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.317e-06 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498364+0000", 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5930000000000002e-06 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498369+0000", 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3630000000000002e-06 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500488+0000", 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021187699999999999 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500492+0000", 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:00:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529179+0000", 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286873590000001 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529184+0000", 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131376 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.617968875000001, 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030817713, 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131376 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.206 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498383+0000", 2024-09-14T07:00:30.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498385+0000", 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.091e-06 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498391+0000", 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1070000000000001e-06 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498396+0000", 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4670000000000003e-06 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500514+0000", 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021177449999999999 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500518+0000", 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8770000000000003e-06 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529195+0000", 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028676938 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529200+0000", 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131377 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:30.210 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 13.617929350000001, 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308008749999999, 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131377 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498425+0000", 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5620000000000002e-06 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498431+0000", 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2539999999999998e-06 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498436+0000", 2024-09-14T07:00:30.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.31e-06 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500539+0000", 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021025459999999998 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500543+0000", 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.597e-06 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529218+0000", 2024-09-14T07:00:30.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286753930000001 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529223+0000", 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.758e-06 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.215 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:30.217 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:30.217 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:30.216+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:30.286 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T07:00:30.294 INFO:teuthology.orchestra.run.smithi007.stdout:3924 trunc from 0xcbb6f34 to 0xd114d7d 2024-09-14T07:00:30.301 INFO:teuthology.orchestra.run.smithi007.stdout:3925 write 0x5a94e44 thru 0x5a9867d (0x383a bytes) 2024-09-14T07:00:30.312 INFO:teuthology.orchestra.run.smithi007.stdout:3927 writesame 0xdc7a19a thru 0xdc83504 data_size 0x1(0x936b bytes) 2024-09-14T07:00:30.312 INFO:teuthology.orchestra.run.smithi007.stdout:3928 read 0x3800440 thru 0x3805168 (0x4d29 bytes) 2024-09-14T07:00:30.312 INFO:teuthology.orchestra.run.smithi007.stdout:3929 write 0x370d499 thru 0x3710a34 (0x359c bytes) 2024-09-14T07:00:30.315 INFO:teuthology.orchestra.run.smithi007.stdout:3930 read 0x1a25713 thru 0x1a27c13 (0x2501 bytes) 2024-09-14T07:00:30.315 INFO:teuthology.orchestra.run.smithi007.stdout:3932 write 0xb5f54f3 thru 0xb60017f (0xac8d bytes) 2024-09-14T07:00:30.317 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:30.316+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:30.367 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:30.368 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:30.368 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:30.368 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:30.376 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:30.419 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:30.419+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:30.478 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:30.478 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:30.478 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:30.479 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:30.479 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:30.487 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:30.519 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:30.517+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:30.620 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:30.619+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.464145061000004, 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:00:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.812801956000001, 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:00:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:00:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.812786488, 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:00:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.812775187, 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.806031714, 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:00:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.806019110000001, 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:00:30.653 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:00:30.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805999593999999, 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.658 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:00:30.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805987553000001, 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:00:30.662 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:00:30.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:00:30.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.666 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805967250999998, 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.667 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805954934999999, 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.671 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:00:30.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.675 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805943495000001, 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:30.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:00:30.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805932995999999, 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.680 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.684 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805922803000001, 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:00:30.688 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805907256000001, 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.689 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:00:30.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:30.693 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805877647999999, 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:30.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:00:30.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.697 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805855997999998, 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.698 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:00:30.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:00:30.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.805833758999999, 2024-09-14T07:00:30.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:00:30.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.800237394, 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:00:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 22.800227340999999, 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:00:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:118426 3.b 3:d50c2793:::rbd_data.114f74d20483.000000000000001c:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 18.128845538, 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.95135023100000005, 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 118426 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500745+0000", 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:00:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500748+0000", 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1329999999999999e-06 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500759+0000", 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0475000000000001e-05 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500765+0000", 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500824+0000", 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500830+0000", 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5889999999999999e-06 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450787+0000", 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.94995644099999998 2024-09-14T07:00:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450901+0000", 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114032 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451052+0000", 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015113 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:30.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451746+0000", 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069442799999999997 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452006+0000", 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025975 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452059+0000", 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3090000000000002e-05 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:30.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452095+0000", 2024-09-14T07:00:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5395999999999998e-05 2024-09-14T07:00:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:30.722 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:30.722 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:30.723 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:30.724 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:30.723+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:30.824 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:30.823+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:30.867 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:30.867 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:30.868 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:30.868 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:30.875 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:30.960 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:30.959+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:31.019 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:31.019 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:31.019 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:31.020 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:31.020 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:31.025 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:31.024+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:31.028 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:31.126 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:31.125+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:31.173 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:31.173 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:31.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:31.173 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:31.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.173 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:00:31.173 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:31.173 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.407464583999996, 2024-09-14T07:00:31.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:00:31.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:00:31.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:00:31.177 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.334345317999997, 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.178 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.183 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.334088665000003, 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.184 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:00:31.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:00:31.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.307801591, 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:00:31.188 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:31.192 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.307676977, 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:31.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:00:31.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.813057815000001, 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.197 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.201 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.538655155000001, 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:00:31.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.206 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.206 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.206 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:00:31.206 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:31.206 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.544039544999997, 2024-09-14T07:00:31.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:00:31.206 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.207 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.207 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.207 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:31.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:31.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:31.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:00:31.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:00:31.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:00:31.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:00:31.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:00:31.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:00:31.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:00:31.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:00:31.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:00:31.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:00:31.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:00:31.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:00:31.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:00:31.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:00:31.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:00:31.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.214 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.353867078999997, 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:00:31.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:00:31.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:00:31.219 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.304733788999997, 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.223 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.936514825000003, 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:00:31.228 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.815945886000002, 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.232 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.685030005000002, 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:00:31.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.655992742999999, 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:00:31.237 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:00:31.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.630223321000003, 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:00:31.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:00:31.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.246 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.596865178000002, 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:00:31.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.432657397, 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.251 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:00:31.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:00:31.255 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.421278137000002, 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:00:31.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.421230442999999, 2024-09-14T07:00:31.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:31.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:31.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:00:31.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:31.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 46.420888118000001, 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.270 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:31.274 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:31.275 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:31.273+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:31.295 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.295 INFO:teuthology.orchestra.run.smithi007.stdout:3934 read 0x338a8af thru 0x33957d1 (0xaf23 bytes) 2024-09-14T07:00:31.295 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.296 INFO:teuthology.orchestra.run.smithi007.stdout:3937 writesame 0x78fc8cf thru 0x790352d data_size 0x1(0x6c5f bytes) 2024-09-14T07:00:31.331 INFO:teuthology.orchestra.run.smithi007.stdout:3938 trunc from 0xdc83505 to 0xdef7d3 2024-09-14T07:00:31.345 INFO:teuthology.orchestra.run.smithi007.stdout:3940 punch from 0xbffbf9 to 0xc0d232, (0xd639 bytes) 2024-09-14T07:00:31.345 INFO:teuthology.orchestra.run.smithi007.stdout:3941 read 0x9bd801 thru 0x9bf4a0 (0x1ca0 bytes) 2024-09-14T07:00:31.345 INFO:teuthology.orchestra.run.smithi007.stdout:3942 read 0x7a3e9b thru 0x7a450d (0x673 bytes) 2024-09-14T07:00:31.369 INFO:teuthology.orchestra.run.smithi007.stdout:3943 writesame 0xda466da thru 0xda51c3b data_size 0x6(0xb562 bytes) 2024-09-14T07:00:31.369 INFO:teuthology.orchestra.run.smithi007.stdout:3946 write 0x43c1a73 thru 0x43c2a34 (0xfc2 bytes) 2024-09-14T07:00:31.372 INFO:teuthology.orchestra.run.smithi007.stdout:3948 write 0xd322384 thru 0xd3277b9 (0x5436 bytes) 2024-09-14T07:00:31.375 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:31.374+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:3949 read 0xcd8b6dd thru 0xcd98377 (0xcc9b bytes) 2024-09-14T07:00:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.375 INFO:teuthology.orchestra.run.smithi007.stdout:3951 writesame 0x158b958 thru 0x15972c5 data_size 0x2f(0xb96e bytes) 2024-09-14T07:00:31.378 INFO:teuthology.orchestra.run.smithi007.stdout:3952 write 0xb16db88 thru 0xb1734f8 (0x5971 bytes) 2024-09-14T07:00:31.381 INFO:teuthology.orchestra.run.smithi007.stdout:3953 read 0x76b3f78 thru 0x76c209f (0xe128 bytes) 2024-09-14T07:00:31.381 INFO:teuthology.orchestra.run.smithi007.stdout:3954 read 0x901f0c9 thru 0x902a98d (0xb8c5 bytes) 2024-09-14T07:00:31.397 INFO:teuthology.orchestra.run.smithi007.stdout:3955 trunc from 0xda51c3c to 0x1df2d0a 2024-09-14T07:00:31.406 INFO:teuthology.orchestra.run.smithi007.stdout:3956 punch from 0x197e541 to 0x198bff8, (0xdab7 bytes) 2024-09-14T07:00:31.412 INFO:teuthology.orchestra.run.smithi007.stdout:3957 write 0x2325757 thru 0x232ab88 (0x5432 bytes) 2024-09-14T07:00:31.423 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:31.423 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:31.423 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:31.423 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:31.432 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:31.435 INFO:teuthology.orchestra.run.smithi007.stdout:3958 write 0xe972eeb thru 0xe9829db (0xfaf1 bytes) 2024-09-14T07:00:31.439 INFO:teuthology.orchestra.run.smithi007.stdout:3960 read 0x5c38747 thru 0x5c3c01b (0x38d5 bytes) 2024-09-14T07:00:31.439 INFO:teuthology.orchestra.run.smithi007.stdout:3963 write 0x16da2f8 thru 0x16dc80a (0x2513 bytes) 2024-09-14T07:00:31.442 INFO:teuthology.orchestra.run.smithi007.stdout:3964 writesame 0x270b668 thru 0x27118bd data_size 0x2(0x6256 bytes) 2024-09-14T07:00:31.451 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.451 INFO:teuthology.orchestra.run.smithi007.stdout:3967 punch from 0xe501210 to 0xe502b2f, (0x191f bytes) 2024-09-14T07:00:31.451 INFO:teuthology.orchestra.run.smithi007.stdout:3970 punch from 0x2d67f9f to 0x2d6e29c, (0x62fd bytes) 2024-09-14T07:00:31.451 INFO:teuthology.orchestra.run.smithi007.stdout:3971 read 0xbe9ee5e thru 0xbeaa345 (0xb4e8 bytes) 2024-09-14T07:00:31.451 INFO:teuthology.orchestra.run.smithi007.stdout:3972 trunc from 0xe9829dc to 0xdd89689 2024-09-14T07:00:31.461 INFO:teuthology.orchestra.run.smithi007.stdout:3974 write 0xbdf18a7 thru 0xbdf606f (0x47c9 bytes) 2024-09-14T07:00:31.465 INFO:teuthology.orchestra.run.smithi007.stdout:3976 write 0x62dbe06 thru 0x62e86ee (0xc8e9 bytes) 2024-09-14T07:00:31.468 INFO:teuthology.orchestra.run.smithi007.stdout:3978 write 0x5077f4 thru 0x50f70f (0x7f1c bytes) 2024-09-14T07:00:31.477 INFO:teuthology.orchestra.run.smithi007.stdout:3979 trunc from 0xdd89689 to 0xaf7b72e 2024-09-14T07:00:31.484 INFO:teuthology.orchestra.run.smithi007.stdout:3980 punch from 0x315d49b to 0x31676e9, (0xa24e bytes) 2024-09-14T07:00:31.485 INFO:teuthology.orchestra.run.smithi007.stdout:3981 punch from 0x35ac299 to 0x35b7e42, (0xbba9 bytes) 2024-09-14T07:00:31.485 INFO:teuthology.orchestra.run.smithi007.stdout:3982 write 0x6b64937 thru 0x6b716a8 (0xcd72 bytes) 2024-09-14T07:00:31.488 INFO:teuthology.orchestra.run.smithi007.stdout:3983 writesame 0x8aa243f thru 0x8aae2e3 data_size 0x2b(0xbea5 bytes) 2024-09-14T07:00:31.492 INFO:teuthology.orchestra.run.smithi007.stdout:3985 read 0x356716 thru 0x3584ce (0x1db9 bytes) 2024-09-14T07:00:31.492 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.492 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.493 INFO:teuthology.orchestra.run.smithi007.stdout:3989 read 0x9361480 thru 0x93642bb (0x2e3c bytes) 2024-09-14T07:00:31.493 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.493 INFO:teuthology.orchestra.run.smithi007.stdout:3992 write 0x2c4a783 thru 0x2c54e5c (0xa6da bytes) 2024-09-14T07:00:31.495 INFO:teuthology.orchestra.run.smithi007.stdout:3993 punch from 0x5507ee9 to 0x5509fe1, (0x20f8 bytes) 2024-09-14T07:00:31.495 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.495 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.495 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.504 INFO:teuthology.orchestra.run.smithi007.stdout:4000 trunc from 0xaf7b72e to 0x36417bb 2024-09-14T07:00:31.514 INFO:teuthology.orchestra.run.smithi007.stdout:4001 read 0xcf1b35 thru 0xcf7ca2 (0x616e bytes) 2024-09-14T07:00:31.518 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:31.518+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:31.524 INFO:teuthology.orchestra.run.smithi007.stdout:4002 trunc from 0x36417bb to 0xbf7187e 2024-09-14T07:00:31.529 INFO:teuthology.orchestra.run.smithi007.stdout:4003 read 0x2097503 thru 0x2099dee (0x28ec bytes) 2024-09-14T07:00:31.529 INFO:teuthology.orchestra.run.smithi007.stdout:4004 writesame 0x328ff37 thru 0x3298da0 data_size 0x1(0x8e6a bytes) 2024-09-14T07:00:31.549 INFO:teuthology.orchestra.run.smithi007.stdout:4005 trunc from 0xbf7187e to 0xda22894 2024-09-14T07:00:31.554 INFO:teuthology.orchestra.run.smithi007.stdout:4007 writesame 0x284b942 thru 0x285547c data_size 0x7(0x9b3b bytes) 2024-09-14T07:00:31.564 INFO:teuthology.orchestra.run.smithi007.stdout:4008 writesame 0xde37514 thru 0xde45261 data_size 0xd(0xdd4e bytes) 2024-09-14T07:00:31.564 INFO:teuthology.orchestra.run.smithi007.stdout:4010 read 0x55bee9 thru 0x56a7b1 (0xe8c9 bytes) 2024-09-14T07:00:31.565 INFO:teuthology.orchestra.run.smithi007.stdout:4012 write 0x941a4db thru 0x94276ad (0xd1d3 bytes) 2024-09-14T07:00:31.568 INFO:teuthology.orchestra.run.smithi007.stdout:4013 read 0x22939 thru 0x2dfac (0xb674 bytes) 2024-09-14T07:00:31.569 INFO:teuthology.orchestra.run.smithi007.stdout:4014 punch from 0xa144fc8 to 0xa152598, (0xd5d0 bytes) 2024-09-14T07:00:31.569 INFO:teuthology.orchestra.run.smithi007.stdout:4015 writesame 0x4a8853d thru 0x4a915ed data_size 0x1(0x90b1 bytes) 2024-09-14T07:00:31.578 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:31.578 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:31.578 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:31.578 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:31.578 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:31.578 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:31.576+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:31.587 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:31.588 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.588 INFO:teuthology.orchestra.run.smithi007.stdout:4017 read 0xce7d2f2 thru 0xce8cfe8 (0xfcf7 bytes) 2024-09-14T07:00:31.588 INFO:teuthology.orchestra.run.smithi007.stdout:4021 read 0x1ec154a thru 0x1ec929d (0x7d54 bytes) 2024-09-14T07:00:31.588 INFO:teuthology.orchestra.run.smithi007.stdout:4022 write 0x7880b22 thru 0x7882576 (0x1a55 bytes) 2024-09-14T07:00:31.592 INFO:teuthology.orchestra.run.smithi007.stdout:4024 punch from 0xda52865 to 0xda54414, (0x1baf bytes) 2024-09-14T07:00:31.592 INFO:teuthology.orchestra.run.smithi007.stdout:4025 writesame 0xbdc77d0 thru 0xbdce7fb data_size 0x6(0x702c bytes) 2024-09-14T07:00:31.603 INFO:teuthology.orchestra.run.smithi007.stdout:4026 trunc from 0xde45262 to 0x7f2dde1 2024-09-14T07:00:31.612 INFO:teuthology.orchestra.run.smithi007.stdout:4027 flatten 2024-09-14T07:00:31.624 INFO:teuthology.orchestra.run.smithi007.stdout:4028 punch from 0x150c8c7 to 0x1515cca, (0x9403 bytes) 2024-09-14T07:00:31.624 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.624 INFO:teuthology.orchestra.run.smithi007.stdout:4030 read 0x281efb1 thru 0x2822990 (0x39e0 bytes) 2024-09-14T07:00:31.624 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.633 INFO:teuthology.orchestra.run.smithi007.stdout:4033 writesame 0xb6c2279 thru 0xb6c50c8 data_size 0xd(0x2e50 bytes) 2024-09-14T07:00:31.633 INFO:teuthology.orchestra.run.smithi007.stdout:4034 punch from 0x9fecad8 to 0x9ff09c0, (0x3ee8 bytes) 2024-09-14T07:00:31.633 INFO:teuthology.orchestra.run.smithi007.stdout:4035 write 0x54fc999 thru 0x5508dd0 (0xc438 bytes) 2024-09-14T07:00:31.647 INFO:teuthology.orchestra.run.smithi007.stdout:4036 writesame 0xe7efefd thru 0xe7fde87 data_size 0x1(0xdf8b bytes) 2024-09-14T07:00:31.647 INFO:teuthology.orchestra.run.smithi007.stdout:4039 write 0x61d2f73 thru 0x61e0ddb (0xde69 bytes) 2024-09-14T07:00:31.651 INFO:teuthology.orchestra.run.smithi007.stdout:4040 read 0xb922bd4 thru 0xb9246e3 (0x1b10 bytes) 2024-09-14T07:00:31.651 INFO:teuthology.orchestra.run.smithi007.stdout:4042 write 0xb657310 thru 0xb65b51a (0x420b bytes) 2024-09-14T07:00:31.654 INFO:teuthology.orchestra.run.smithi007.stdout:4044 write 0x2036e4f thru 0x203dc87 (0x6e39 bytes) 2024-09-14T07:00:31.657 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.657 INFO:teuthology.orchestra.run.smithi007.stdout:4046 punch from 0x8b699e6 to 0x8b73588, (0x9ba2 bytes) 2024-09-14T07:00:31.657 INFO:teuthology.orchestra.run.smithi007.stdout:4047 read 0x98687e3 thru 0x986f46f (0x6c8d bytes) 2024-09-14T07:00:31.657 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.657 INFO:teuthology.orchestra.run.smithi007.stdout:4049 read 0x4408250 thru 0x4409abb (0x186c bytes) 2024-09-14T07:00:31.658 INFO:teuthology.orchestra.run.smithi007.stdout:4050 write 0x416e7ae thru 0x4174a64 (0x62b7 bytes) 2024-09-14T07:00:31.660 INFO:teuthology.orchestra.run.smithi007.stdout:4051 read 0x42a9c0d thru 0x42b21bd (0x85b1 bytes) 2024-09-14T07:00:31.660 INFO:teuthology.orchestra.run.smithi007.stdout:4052 writesame 0xc5a6434 thru 0xc5a6b7e data_size 0x1(0x74b bytes) 2024-09-14T07:00:31.663 INFO:teuthology.orchestra.run.smithi007.stdout:4053 write 0x37f40ea thru 0x37f599f (0x18b6 bytes) 2024-09-14T07:00:31.666 INFO:teuthology.orchestra.run.smithi007.stdout:4054 writesame 0x911ade thru 0x919ff6 data_size 0x1(0x8519 bytes) 2024-09-14T07:00:31.679 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:31.678+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:31.682 INFO:teuthology.orchestra.run.smithi007.stdout:4056 write 0x40d73 thru 0x4b5db (0xa869 bytes) 2024-09-14T07:00:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.683 INFO:teuthology.orchestra.run.smithi007.stdout:4060 writesame 0x9f2fe4e thru 0x9f3c13b data_size 0x3(0xc2ee bytes) 2024-09-14T07:00:31.693 INFO:teuthology.orchestra.run.smithi007.stdout:4062 read 0xd118909 thru 0xd1264e3 (0xdbdb bytes) 2024-09-14T07:00:31.693 INFO:teuthology.orchestra.run.smithi007.stdout:4063 punch from 0xe2b7969 to 0xe2c3830, (0xbec7 bytes) 2024-09-14T07:00:31.694 INFO:teuthology.orchestra.run.smithi007.stdout:4064 trunc from 0xe7fde88 to 0xdc85aea 2024-09-14T07:00:31.695 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:31.696 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:31.696 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.6944818633987457 2024-09-14T07:00:31.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 0.6944818633987457 2024-09-14T07:00:31.700 INFO:teuthology.orchestra.run.smithi007.stdout:4068 read 0x87b06bb thru 0x87b6efd (0x6843 bytes) 2024-09-14T07:00:31.700 INFO:teuthology.orchestra.run.smithi007.stdout:4069 write 0xb74a42d thru 0xb757954 (0xd528 bytes) 2024-09-14T07:00:31.701 INFO:teuthology.orchestra.run.smithi007.stdout:4070 writesame 0xd9b4e05 thru 0xd9bfc96 data_size 0xa(0xae92 bytes) 2024-09-14T07:00:31.706 INFO:teuthology.orchestra.run.smithi007.stdout:4071 write 0x2a08fd1 thru 0x2a14fcf (0xbfff bytes) 2024-09-14T07:00:31.711 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.711 INFO:teuthology.orchestra.run.smithi007.stdout:4074 read 0xd48af92 thru 0xd49ab23 (0xfb92 bytes) 2024-09-14T07:00:31.711 INFO:teuthology.orchestra.run.smithi007.stdout:4076 read 0xa554df thru 0xa5e1ac (0x8cce bytes) 2024-09-14T07:00:31.711 INFO:teuthology.orchestra.run.smithi007.stdout:4077 punch from 0x33363ff to 0x3344fb6, (0xebb7 bytes) 2024-09-14T07:00:31.711 INFO:teuthology.orchestra.run.smithi007.stdout:4078 read 0xb0228b5 thru 0xb02bad6 (0x9222 bytes) 2024-09-14T07:00:31.711 INFO:teuthology.orchestra.run.smithi007.stdout:4080 punch from 0x22c3ae0 to 0x22d0231, (0xc751 bytes) 2024-09-14T07:00:31.711 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.711 INFO:teuthology.orchestra.run.smithi007.stdout:4082 write 0x2ec2d2c thru 0x2ec623a (0x350f bytes) 2024-09-14T07:00:31.729 INFO:teuthology.orchestra.run.smithi007.stdout:4083 trunc from 0xdc85aea to 0xc0a7a4 2024-09-14T07:00:31.738 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:31.738 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:31.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:31.738 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:31.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.738 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8489 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8208384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066383365, 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033127139, 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8489 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670276+0000", 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:31.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670277+0000", 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6109999999999999e-06 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670283+0000", 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7409999999999998e-06 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670287+0000", 2024-09-14T07:00:31.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1679999999999997e-06 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703311+0000", 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033023640999999999 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703352+0000", 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1168999999999998e-05 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703402+0000", 2024-09-14T07:00:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0424999999999999e-05 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8490 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066365781999998, 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033303263, 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8490 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670295+0000", 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670300+0000", 2024-09-14T07:00:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5369999999999997e-06 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670304+0000", 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8789999999999996e-06 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703433+0000", 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033128697999999998 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703510+0000", 2024-09-14T07:00:31.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6662000000000006e-05 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703596+0000", 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6614999999999997e-05 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8491 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8454144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066348152000003, 2024-09-14T07:00:31.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033389611999999999, 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8491 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4900000000000001e-07 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670317+0000", 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.19e-06 2024-09-14T07:00:31.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670323+0000", 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670331+0000", 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1280000000000008e-06 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703611+0000", 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033279557000000001 2024-09-14T07:00:31.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703650+0000", 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9410999999999999e-05 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703700+0000", 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0117000000000001e-05 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8492 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8568832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066320101999999, 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033561793999999999, 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8492 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:00:31.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670341+0000", 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8700000000000001e-06 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670347+0000", 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000004e-06 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670351+0000", 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4660000000000001e-06 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703731+0000", 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033379502999999998 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703812+0000", 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1192000000000005e-05 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703900+0000", 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8382999999999997e-05 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8493 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8683520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066301604000003, 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033663516999999997, 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.756 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8493 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670359+0000", 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.685e-06 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670365+0000", 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.57e-06 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670369+0000", 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4460000000000003e-06 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703928+0000", 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033558530000000003 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703970+0000", 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1909999999999997e-05 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704021+0000", 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0988e-05 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.760 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8494 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8798208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066272007999999, 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033828579999999997, 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8494 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670389+0000", 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.64e-06 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670393+0000", 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3470000000000001e-06 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670396+0000", 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6369999999999999e-06 2024-09-14T07:00:31.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704051+0000", 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033655293000000003 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704128+0000", 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7453000000000005e-05 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704215+0000", 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6884000000000004e-05 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8495 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8912896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066258458, 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033930725000000002, 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.765 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8495 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:31.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670401+0000", 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.15e-07 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670402+0000", 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.850000000000001e-07 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670405+0000", 2024-09-14T07:00:31.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.225e-06 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670407+0000", 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.34e-06 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704240+0000", 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033833019999999998 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704281+0000", 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1332999999999999e-05 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704331+0000", 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9506999999999999e-05 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8496 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9027584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066247134000001, 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034114617, 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8496 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:31.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670413+0000", 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8999999999999995e-07 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670416+0000", 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2729999999999998e-06 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670418+0000", 2024-09-14T07:00:31.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3709999999999998e-06 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704362+0000", 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033943917999999997 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704439+0000", 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6922999999999995e-05 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704526+0000", 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7035999999999998e-05 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8497 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9142272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066237246, 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034220315000000001, 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.774 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8497 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670422+0000", 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670426+0000", 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5029999999999999e-06 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670430+0000", 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-06 2024-09-14T07:00:31.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670432+0000", 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704552+0000", 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034120277999999997 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704592+0000", 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9884e-05 2024-09-14T07:00:31.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704642+0000", 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9623999999999998e-05 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8498 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9256960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066223483000002, 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034407434000000001, 2024-09-14T07:00:31.778 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8498 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670436+0000", 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:00:31.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670439+0000", 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670442+0000", 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3120000000000001e-06 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704673+0000", 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034231175000000003 2024-09-14T07:00:31.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704749+0000", 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6069999999999995e-05 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704843+0000", 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3622000000000006e-05 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8499 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9371648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066213963000003, 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034523731000000002, 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8499 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2000000000000001e-07 2024-09-14T07:00:31.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670446+0000", 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670449+0000", 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809999999999998e-06 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670455+0000", 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7930000000000003e-06 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704869+0000", 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034413931000000002 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704910+0000", 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1032999999999998e-05 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704968+0000", 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8897999999999998e-05 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8500 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9486336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066200686000002, 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034708731, 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.787 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8500 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:31.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3300000000000001e-07 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670459+0000", 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670462+0000", 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1350000000000001e-06 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670470+0000", 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7400000000000004e-06 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705002+0000", 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034531470000000002 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705079+0000", 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7472999999999995e-05 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705167+0000", 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7750000000000005e-05 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8501 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9601024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:31.791 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066182257000001, 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034806278000000003, 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8501 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:31.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670477+0000", 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.44e-07 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670479+0000", 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7740000000000001e-06 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670485+0000", 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9150000000000001e-06 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670489+0000", 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5469999999999998e-06 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705191+0000", 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034701896000000003 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705234+0000", 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2394000000000001e-05 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705283+0000", 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9208e-05 2024-09-14T07:00:31.795 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8502 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9715712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066161917999999, 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034979270999999999, 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.796 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8502 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670506+0000", 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9740000000000008e-06 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670512+0000", 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670517+0000", 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7049999999999998e-06 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705313+0000", 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034795858999999998 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705391+0000", 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8166999999999999e-05 2024-09-14T07:00:31.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705476+0000", 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5285999999999996e-05 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8503 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9830400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066135496000001, 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053094, 2024-09-14T07:00:31.800 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8503 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670524+0000", 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670525+0000", 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.581e-06 2024-09-14T07:00:31.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670531+0000", 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8969999999999996e-06 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670536+0000", 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4630000000000003e-06 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705488+0000", 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034952451000000002 2024-09-14T07:00:31.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705526+0000", 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8396999999999998e-05 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705576+0000", 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9926000000000003e-05 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8504 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9945088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066117157999997, 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035228601999999998, 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8504 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:31.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670543+0000", 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5820000000000001e-06 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670549+0000", 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670554+0000", 2024-09-14T07:00:31.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3699999999999997e-06 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705607+0000", 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053594 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705685+0000", 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8102999999999994e-05 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705770+0000", 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4678000000000005e-05 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8505 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10059776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066099371, 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035318548999999998, 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.809 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8505 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:31.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670560+0000", 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670561+0000", 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670568+0000", 2024-09-14T07:00:31.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9789999999999996e-06 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670572+0000", 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2509999999999996e-06 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705784+0000", 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035211724 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705827+0000", 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3276999999999999e-05 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705878+0000", 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0488000000000002e-05 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8506 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10174464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066079285000001, 2024-09-14T07:00:31.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035495512999999999, 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8506 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670580+0000", 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670582+0000", 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:31.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670587+0000", 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.767e-06 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670592+0000", 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2819999999999998e-06 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705909+0000", 2024-09-14T07:00:31.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.03531753 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705987+0000", 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7787999999999995e-05 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706075+0000", 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7998999999999996e-05 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8507 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10289152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066061458999997, 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035582134000000001, 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8507 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.818 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670598+0000", 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9900000000000002e-07 2024-09-14T07:00:31.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670599+0000", 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6029999999999999e-06 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670609+0000", 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.004e-05 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670614+0000", 2024-09-14T07:00:31.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0749999999999997e-06 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706089+0000", 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035474358999999997 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706129+0000", 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0204999999999999e-05 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706179+0000", 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0553000000000002e-05 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8508 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10403840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 48.066037829999999, 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035743559000000001, 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:31.822 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8508 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2800000000000002e-07 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670623+0000", 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670629+0000", 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.02e-06 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670634+0000", 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6280000000000004e-06 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706200+0000", 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035566564000000002 2024-09-14T07:00:31.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706276+0000", 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5801000000000002e-05 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706364+0000", 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8176000000000004e-05 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.826 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:31.827 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:31.827 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:31.829 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:31.827+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:31.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:31.830 INFO:teuthology.orchestra.run.smithi007.stdout:4084 read 0xb8b1a8 thru 0xb943f0 (0x9249 bytes) 2024-09-14T07:00:31.830 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.830 INFO:teuthology.orchestra.run.smithi007.stdout:4087 write 0xc30d735 thru 0xc31a49b (0xcd67 bytes) 2024-09-14T07:00:31.830 INFO:teuthology.orchestra.run.smithi007.stdout:4088 writesame 0xd6b953e thru 0xd6bb8c7 data_size 0x1(0x238a bytes) 2024-09-14T07:00:31.830 INFO:teuthology.orchestra.run.smithi007.stdout:4089 read 0x81f47c1 thru 0x81f8f06 (0x4746 bytes) 2024-09-14T07:00:31.830 INFO:teuthology.orchestra.run.smithi007.stdout:4091 writesame 0xdf06b5f thru 0xdf16afb data_size 0x1(0xff9d bytes) 2024-09-14T07:00:31.831 INFO:teuthology.orchestra.run.smithi007.stdout:4092 write 0x10c734b thru 0x10d2a91 (0xb747 bytes) 2024-09-14T07:00:31.831 INFO:teuthology.orchestra.run.smithi007.stdout:4093 read 0xd1c0b49 thru 0xd1cb008 (0xa4c0 bytes) 2024-09-14T07:00:31.831 INFO:teuthology.orchestra.run.smithi007.stdout:4094 writesame 0x5e46f58 thru 0x5e4bcbc data_size 0x1(0x4d65 bytes) 2024-09-14T07:00:31.831 INFO:teuthology.orchestra.run.smithi007.stdout:4096 trunc from 0xdf16afc to 0x8b555b9 2024-09-14T07:00:31.831 INFO:teuthology.orchestra.run.smithi007.stdout:4097 read 0x1da9e9 thru 0x1de55a (0x3b72 bytes) 2024-09-14T07:00:31.831 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.831 INFO:teuthology.orchestra.run.smithi007.stdout:4099 trunc from 0x8b555b9 to 0x4e93632 2024-09-14T07:00:31.831 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.837 INFO:teuthology.orchestra.run.smithi007.stdout:4101 write 0xbb5fe1b thru 0xbb65dc3 (0x5fa9 bytes) 2024-09-14T07:00:31.842 INFO:teuthology.orchestra.run.smithi007.stdout:4102 read 0xaf297d2 thru 0xaf2b8de (0x210d bytes) 2024-09-14T07:00:31.842 INFO:teuthology.orchestra.run.smithi007.stdout:4105 punch from 0x90f1dd5 to 0x90fd9b3, (0xbbde bytes) 2024-09-14T07:00:31.842 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.843 INFO:teuthology.orchestra.run.smithi007.stdout:4109 trunc from 0xbb65dc4 to 0xa614450 2024-09-14T07:00:31.855 INFO:teuthology.orchestra.run.smithi007.stdout:4110 read 0x3c06213 thru 0x3c0daa7 (0x7895 bytes) 2024-09-14T07:00:31.855 INFO:teuthology.orchestra.run.smithi007.stdout:4111 write 0x8526bc1 thru 0x85328f5 (0xbd35 bytes) 2024-09-14T07:00:31.867 INFO:teuthology.orchestra.run.smithi007.stdout:4112 write 0xbfcc33e thru 0xbfd5955 (0x9618 bytes) 2024-09-14T07:00:31.871 INFO:teuthology.orchestra.run.smithi007.stdout:4113 read 0xa35570d thru 0xa36548a (0xfd7e bytes) 2024-09-14T07:00:31.871 INFO:teuthology.orchestra.run.smithi007.stdout:4114 read 0x874a0f5 thru 0x874f0a0 (0x4fac bytes) 2024-09-14T07:00:31.871 INFO:teuthology.orchestra.run.smithi007.stdout:4115 writesame 0x15d02dd thru 0x15d4966 data_size 0x1(0x468a bytes) 2024-09-14T07:00:31.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:31.881 INFO:teuthology.orchestra.run.smithi007.stdout:4116 write 0x71e3cd0 thru 0x71e6e61 (0x3192 bytes) 2024-09-14T07:00:31.892 INFO:teuthology.orchestra.run.smithi007.stdout:4118 write 0xc2eb7bb thru 0xc2f0325 (0x4b6b bytes) 2024-09-14T07:00:31.906 INFO:teuthology.orchestra.run.smithi007.stdout:4119 write 0xe7fb1e6 thru 0xe801fbe (0x6dd9 bytes) 2024-09-14T07:00:31.913 INFO:teuthology.orchestra.run.smithi007.stdout:4120 write 0xc1e5041 thru 0xc1f1ccf (0xcc8f bytes) 2024-09-14T07:00:31.916 INFO:teuthology.orchestra.run.smithi007.stdout:4122 write 0x7df768e thru 0x7dfbd92 (0x4705 bytes) 2024-09-14T07:00:31.920 INFO:teuthology.orchestra.run.smithi007.stdout:4123 punch from 0xbddb885 to 0xbde56d8, (0x9e53 bytes) 2024-09-14T07:00:31.920 INFO:teuthology.orchestra.run.smithi007.stdout:4124 read 0x31aa7b0 thru 0x31b42de (0x9b2f bytes) 2024-09-14T07:00:31.921 INFO:teuthology.orchestra.run.smithi007.stdout:4126 writesame 0xc9443d5 thru 0xc94c714 data_size 0x64(0x8340 bytes) 2024-09-14T07:00:31.924 INFO:teuthology.orchestra.run.smithi007.stdout:4127 punch from 0xcbb20aa to 0xcbbb95d, (0x98b3 bytes) 2024-09-14T07:00:31.924 INFO:teuthology.orchestra.run.smithi007.stdout:4128 write 0x975737a thru 0x975b124 (0x3dab bytes) 2024-09-14T07:00:31.928 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:31.928+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:31.935 INFO:teuthology.orchestra.run.smithi007.stdout:4129 trunc from 0xe801fbf to 0x8ab6649 2024-09-14T07:00:31.950 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.950 INFO:teuthology.orchestra.run.smithi007.stdout:4132 punch from 0x14c44fd to 0x14d040f, (0xbf12 bytes) 2024-09-14T07:00:31.950 INFO:teuthology.orchestra.run.smithi007.stdout:4133 read 0x6611f83 thru 0x6612d67 (0xde5 bytes) 2024-09-14T07:00:31.950 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:31.955 INFO:teuthology.orchestra.run.smithi007.stdout:4137 trunc from 0x8ab6649 to 0x48ea92b 2024-09-14T07:00:31.960 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:31.960 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:31.960 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:31.960 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:31.969 INFO:teuthology.orchestra.run.smithi007.stdout:4140 punch from 0x42a1bd7 to 0x42ad2d2, (0xb6fb bytes) 2024-09-14T07:00:31.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:31.977 INFO:teuthology.orchestra.run.smithi007.stdout:4141 write 0x5a0d9c3 thru 0x5a16395 (0x89d3 bytes) 2024-09-14T07:00:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:4142 writesame 0xabe8bb9 thru 0xabeaa19 data_size 0xb(0x1e61 bytes) 2024-09-14T07:00:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:4149 read 0x487c75e thru 0x4883f11 (0x77b4 bytes) 2024-09-14T07:00:31.994 INFO:teuthology.orchestra.run.smithi007.stdout:4150 writesame 0x6714169 thru 0x6720997 data_size 0x1(0xc82f bytes) 2024-09-14T07:00:32.024 INFO:teuthology.orchestra.run.smithi007.stdout:4151 read 0x5e70a6a thru 0x5e7856c (0x7b03 bytes) 2024-09-14T07:00:32.024 INFO:teuthology.orchestra.run.smithi007.stdout:4153 read 0xa3222b2 thru 0xa322a8f (0x7de bytes) 2024-09-14T07:00:32.024 INFO:teuthology.orchestra.run.smithi007.stdout:4154 read 0x917f571 thru 0x918bc15 (0xc6a5 bytes) 2024-09-14T07:00:32.024 INFO:teuthology.orchestra.run.smithi007.stdout:4155 read 0x3a9f9e1 thru 0x3aab6c3 (0xbce3 bytes) 2024-09-14T07:00:32.025 INFO:teuthology.orchestra.run.smithi007.stdout:4156 write 0x4b3e798 thru 0x4b4a12e (0xb997 bytes) 2024-09-14T07:00:32.028 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:32.028 INFO:teuthology.orchestra.run.smithi007.stdout:4158 write 0x70dad7c thru 0x70e0210 (0x5495 bytes) 2024-09-14T07:00:32.029 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:32.029+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:32.032 INFO:teuthology.orchestra.run.smithi007.stdout:4160 write 0x32c6c11 thru 0x32d0fc9 (0xa3b9 bytes) 2024-09-14T07:00:32.035 INFO:teuthology.orchestra.run.smithi007.stdout:4161 punch from 0x85aa991 to 0x85b3563, (0x8bd2 bytes) 2024-09-14T07:00:32.035 INFO:teuthology.orchestra.run.smithi007.stdout:4162 write 0xba1032 thru 0xbae012 (0xcfe1 bytes) 2024-09-14T07:00:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:4165 read 0x754877b thru 0x7553fd9 (0xb85f bytes) 2024-09-14T07:00:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:4166 read 0x3ec0abd thru 0x3ec5493 (0x49d7 bytes) 2024-09-14T07:00:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:4168 read 0x22327f0 thru 0x22359ff (0x3210 bytes) 2024-09-14T07:00:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:4170 write 0x2613ee4 thru 0x26186ce (0x47eb bytes) 2024-09-14T07:00:32.042 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:32.043 INFO:teuthology.orchestra.run.smithi007.stdout:4172 writesame 0x27d6d70 thru 0x27dd4e6 data_size 0x1b(0x6777 bytes) 2024-09-14T07:00:32.046 INFO:teuthology.orchestra.run.smithi007.stdout:4175 read 0x1f4c00f thru 0x1f5403e (0x8030 bytes) 2024-09-14T07:00:32.056 INFO:teuthology.orchestra.run.smithi007.stdout:4176 write 0xc16bab7 thru 0xc1774bb (0xba05 bytes) 2024-09-14T07:00:32.059 INFO:teuthology.orchestra.run.smithi007.stdout:4177 writesame 0x739f7f9 thru 0x73a809c data_size 0x84(0x88a4 bytes) 2024-09-14T07:00:32.064 INFO:teuthology.orchestra.run.smithi007.stdout:4179 read 0x3067724 thru 0x3070dec (0x96c9 bytes) 2024-09-14T07:00:32.064 INFO:teuthology.orchestra.run.smithi007.stdout:4183 read 0xae1d5fd thru 0xae20832 (0x3236 bytes) 2024-09-14T07:00:32.065 INFO:teuthology.orchestra.run.smithi007.stdout:4186 punch from 0x238ba58 to 0x238e7ab, (0x2d53 bytes) 2024-09-14T07:00:32.070 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:32.078 INFO:teuthology.orchestra.run.smithi007.stdout:4189 trunc from 0xc1774bc to 0x13513f5 2024-09-14T07:00:32.093 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:32.096 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:32.097 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:32.103 INFO:teuthology.orchestra.run.smithi007.stdout:4191 write 0x53535a6 thru 0x535e874 (0xb2cf bytes) 2024-09-14T07:00:32.108 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:32.108 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:32.112 INFO:teuthology.orchestra.run.smithi007.stdout:4192 write 0x5811736 thru 0x581e6f5 (0xcfc0 bytes) 2024-09-14T07:00:32.115 INFO:teuthology.orchestra.run.smithi007.stdout:4193 read 0x2139460 thru 0x2140c96 (0x7837 bytes) 2024-09-14T07:00:32.115 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:32.117 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:32.117 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:32.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:32.131 INFO:teuthology.orchestra.run.smithi007.stdout:4195 write 0xd2876e8 thru 0xd28bc13 (0x452c bytes) 2024-09-14T07:00:32.131 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:32.130+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:32.136 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:32.137 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:32.142 INFO:teuthology.orchestra.run.smithi007.stdout:4197 trunc from 0xd28bc14 to 0x5ee23e3 2024-09-14T07:00:32.156 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:32.160 INFO:teuthology.orchestra.run.smithi007.stdout:4200 trunc from 0x5ee23e3 to 0xb1ee374 2024-09-14T07:00:32.166 INFO:teuthology.orchestra.run.smithi007.stdout:4203 read 0x873de3b thru 0x874d9b7 (0xfb7d bytes) 2024-09-14T07:00:32.166 INFO:teuthology.orchestra.run.smithi007.stdout:4204 write 0x30fe782 thru 0x30fe9fc (0x27b bytes) 2024-09-14T07:00:32.178 INFO:teuthology.orchestra.run.smithi007.stdout:4206 trunc from 0xb1ee374 to 0x4fcf44f 2024-09-14T07:00:32.180 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:32.188 INFO:teuthology.orchestra.run.smithi007.stdout:4207 read 0x30d9fa3 thru 0x30df5f1 (0x564f bytes) 2024-09-14T07:00:32.188 INFO:teuthology.orchestra.run.smithi007.stdout:4208 write 0x4c2202 thru 0x4c8b7e (0x697d bytes) 2024-09-14T07:00:32.191 INFO:teuthology.orchestra.run.smithi007.stdout:4209 read 0x26d3248 thru 0x26db70c (0x84c5 bytes) 2024-09-14T07:00:32.206 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:32.206 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:32.209 INFO:teuthology.orchestra.run.smithi007.stdout:4210 write 0xda155f4 thru 0xda19f59 (0x4966 bytes) 2024-09-14T07:00:32.213 INFO:teuthology.orchestra.run.smithi007.stdout:4211 read 0x6447735 thru 0x6456a9b (0xf367 bytes) 2024-09-14T07:00:32.225 INFO:teuthology.orchestra.run.smithi007.stdout:4212 trunc from 0xda19f5a to 0x3be0fbd 2024-09-14T07:00:32.232 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:32.230+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:32.241 INFO:teuthology.orchestra.run.smithi007.stdout:4213 trunc from 0x3be0fbd to 0x88d8443 2024-09-14T07:00:32.250 INFO:teuthology.orchestra.run.smithi007.stdout:4214 trunc from 0x88d8443 to 0x4d89fd5 2024-09-14T07:00:32.257 INFO:teuthology.orchestra.run.smithi007.stdout:4216 punch from 0x517380 to 0x5220ad, (0xad2d bytes) 2024-09-14T07:00:32.257 INFO:teuthology.orchestra.run.smithi007.stdout:4217 read 0x433099b thru 0x433ede1 (0xe447 bytes) 2024-09-14T07:00:32.258 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:32.259 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:32.266 INFO:teuthology.orchestra.run.smithi007.stdout:4218 writesame 0x736288e thru 0x736736c data_size 0x1(0x4adf bytes) 2024-09-14T07:00:32.266 INFO:teuthology.orchestra.run.smithi007.stdout:4219 read 0x710d5bb thru 0x71151d8 (0x7c1e bytes) 2024-09-14T07:00:32.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:32.272 INFO:teuthology.orchestra.run.smithi007.stdout:4220 trunc from 0x736736d to 0xc4569a5 2024-09-14T07:00:32.278 INFO:teuthology.orchestra.run.smithi007.stdout:4221 write 0xa813482 thru 0xa81cb3b (0x96ba bytes) 2024-09-14T07:00:32.281 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:32.281 INFO:teuthology.orchestra.run.smithi007.stdout:4224 read 0xae3d054 thru 0xae49e25 (0xcdd2 bytes) 2024-09-14T07:00:32.281 INFO:teuthology.orchestra.run.smithi007.stdout:4226 clone 27 order 21 su 524288 sc 15 2024-09-14T07:00:32.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:00:32.333 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:32.332+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:32.381 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:32.381 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:32.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:32.434 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:32.432+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:32.498 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:32.498 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:32.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:32.519 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 0.694482 (b1c9) 2024-09-14T07:00:32.533 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:32.533+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:32.634 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:32.632+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:32.658 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:32.658 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:32.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:32.719 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-14T07:00:32.735 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:32.733+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:32.825 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:32.825 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:32.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:32.876 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:32.874+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:32.937 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:32.935+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:32.982 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:32.982 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:32.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:33.075 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:33.073+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:33.138 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:33.138 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:33.138 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:33.138+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:33.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:33.240 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:33.239+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:33.294 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:33.295 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:33.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:33.341 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:33.340+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:33.411 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:33.411 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:33.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:33.441 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:33.441+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:33.524 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T07:00:33.532 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:33.533 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:33.543 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:33.544 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:33.541+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:33.643 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:33.641+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:33.654 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:33.654 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:33.663 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:33.743 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:33.742+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:33.767 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:33.767 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:33.776 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:33.845 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:33.844+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:33.880 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:33.880 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:33.888 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:33.945 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:33.945+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:33.992 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:33.992 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:34.001 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:34.046 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:34.045+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:34.143 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:34.143 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:34.146 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:34.146+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:34.152 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:34.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:34.248 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:34.247+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:34.295 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:34.296 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:34.304 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:34.348 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:34.347+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:34.350 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:34.350 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:34.368 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:34.384 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:34.400 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:34.417 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:34.417 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:34.435 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:34.435 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:34.449 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:34.447+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:34.449 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:34.449 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:34.452 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:34.459 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:34.469 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:34.550 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:34.548+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:34.565 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:34.565 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:34.565 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:34.566 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:34.566 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:34.574 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:34.651 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:34.649+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:34.679 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 73.914192990999993, 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.680 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:00:34.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 55.841073725000001, 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.685 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:00:34.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:00:34.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 55.840817072, 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.691 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:00:34.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 55.814529997999998, 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:00:34.695 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:00:34.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.699 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 55.814405383999997, 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:34.700 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:00:34.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 55.319786221999998, 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:00:34.704 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:00:34.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 54.045383561999998, 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:00:34.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.050767952000001, 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.713 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:00:34.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.860595486000001, 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:00:34.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.811462196000001, 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.722 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:00:34.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.726 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.443243232, 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:00:34.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:00:34.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.322674292999999, 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:00:34.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:00:34.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:34.735 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.191758411999999, 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:00:34.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.162721150000003, 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:00:34.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:00:34.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:34.744 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.136951728, 2024-09-14T07:00:34.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:00:34.745 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.745 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.745 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.745 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:34.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:34.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:00:34.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 50.103593584999999, 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:00:34.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.939385803999997, 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:00:34.756 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:00:34.760 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.928006543999999, 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:00:34.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.927958850000003, 2024-09-14T07:00:34.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:00:34.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.769 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.927616524999998, 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:00:34.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:00:34.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:34.774 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:34.776 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:34.776 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:34.775+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:34.876 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:34.875+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:34.880 INFO:teuthology.orchestra.run.smithi007.stdout:4227 punch from 0x38e350c to 0x38f04cc, (0xcfc0 bytes) 2024-09-14T07:00:34.884 INFO:teuthology.orchestra.run.smithi007.stdout:4228 write 0x2c119d5 thru 0x2c1e385 (0xc9b1 bytes) 2024-09-14T07:00:34.889 INFO:teuthology.orchestra.run.smithi007.stdout:4229 writesame 0x1250af6 thru 0x125efeb data_size 0x3(0xe4f6 bytes) 2024-09-14T07:00:34.901 INFO:teuthology.orchestra.run.smithi007.stdout:4231 writesame 0x1ff1541 thru 0x1ff5b6b data_size 0x17(0x462b bytes) 2024-09-14T07:00:34.905 INFO:teuthology.orchestra.run.smithi007.stdout:4232 write 0x301f60d thru 0x302c244 (0xcc38 bytes) 2024-09-14T07:00:34.909 INFO:teuthology.orchestra.run.smithi007.stdout:4233 write 0x158410f thru 0x158f459 (0xb34b bytes) 2024-09-14T07:00:34.914 INFO:teuthology.orchestra.run.smithi007.stdout:4234 punch from 0x260e683 to 0x261350d, (0x4e8a bytes) 2024-09-14T07:00:34.914 INFO:teuthology.orchestra.run.smithi007.stdout:4235 write 0x3482c6 thru 0x34c277 (0x3fb2 bytes) 2024-09-14T07:00:34.915 INFO:teuthology.orchestra.run.smithi007.stdout:4236 read 0x15a5d9a thru 0x15afaf9 (0x9d60 bytes) 2024-09-14T07:00:34.916 INFO:teuthology.orchestra.run.smithi007.stdout:4237 read 0xa02c880 thru 0xa02edc2 (0x2543 bytes) 2024-09-14T07:00:34.916 INFO:teuthology.orchestra.run.smithi007.stdout:4238 write 0x991810 thru 0x99acae (0x949f bytes) 2024-09-14T07:00:34.920 INFO:teuthology.orchestra.run.smithi007.stdout:4240 read 0x300fc3e thru 0x301770b (0x7ace bytes) 2024-09-14T07:00:34.920 INFO:teuthology.orchestra.run.smithi007.stdout:4241 writesame 0xa1f201c thru 0xa1f3baf data_size 0xa(0x1b94 bytes) 2024-09-14T07:00:34.924 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:34.924 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:34.925 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:34.925 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:34.929 INFO:teuthology.orchestra.run.smithi007.stdout:4242 trunc from 0xc4569a5 to 0x82e6e58 2024-09-14T07:00:34.934 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:34.954 INFO:teuthology.orchestra.run.smithi007.stdout:4243 writesame 0x2a75a7a thru 0x2a82999 data_size 0x10(0xcf20 bytes) 2024-09-14T07:00:34.972 INFO:teuthology.orchestra.run.smithi007.stdout:4244 write 0xd9b7cab thru 0xd9c16dd (0x9a33 bytes) 2024-09-14T07:00:34.974 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:34.974 INFO:teuthology.orchestra.run.smithi007.stdout:4246 read 0xb4b2d7c thru 0xb4bb8bb (0x8b40 bytes) 2024-09-14T07:00:34.974 INFO:teuthology.orchestra.run.smithi007.stdout:4247 trunc from 0xd9c16de to 0xcfe048d 2024-09-14T07:00:34.977 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:34.976+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:34.985 INFO:teuthology.orchestra.run.smithi007.stdout:4248 read 0x1de7ffa thru 0x1df291f (0xa926 bytes) 2024-09-14T07:00:34.985 INFO:teuthology.orchestra.run.smithi007.stdout:4249 read 0xa1cba26 thru 0xa1d8188 (0xc763 bytes) 2024-09-14T07:00:34.985 INFO:teuthology.orchestra.run.smithi007.stdout:4250 punch from 0x3853807 to 0x385814b, (0x4944 bytes) 2024-09-14T07:00:34.985 INFO:teuthology.orchestra.run.smithi007.stdout:4253 read 0x71f78b thru 0x726dc7 (0x763d bytes) 2024-09-14T07:00:34.986 INFO:teuthology.orchestra.run.smithi007.stdout:4254 trunc from 0xcfe048d to 0xc49490b 2024-09-14T07:00:34.997 INFO:teuthology.orchestra.run.smithi007.stdout:4255 trunc from 0xc49490b to 0x7616bd0 2024-09-14T07:00:35.015 INFO:teuthology.orchestra.run.smithi007.stdout:4256 read 0x62fe50d thru 0x62ff2f6 (0xdea bytes) 2024-09-14T07:00:35.015 INFO:teuthology.orchestra.run.smithi007.stdout:4258 write 0x15b691c thru 0x15b6ba0 (0x285 bytes) 2024-09-14T07:00:35.030 INFO:teuthology.orchestra.run.smithi007.stdout:4259 write 0xed99e81 thru 0xed9c613 (0x2793 bytes) 2024-09-14T07:00:35.034 INFO:teuthology.orchestra.run.smithi007.stdout:4262 punch from 0x39fc4f9 to 0x3a09047, (0xcb4e bytes) 2024-09-14T07:00:35.047 INFO:teuthology.orchestra.run.smithi007.stdout:4265 trunc from 0xed9c614 to 0x3d52c37 2024-09-14T07:00:35.051 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:35.051 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:35.051 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:35.051 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:35.051 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:35.062 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:35.067 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.078 INFO:teuthology.orchestra.run.smithi007.stdout:4267 trunc from 0x3d52c37 to 0xe2544e7 2024-09-14T07:00:35.079 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:35.077+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:35.085 INFO:teuthology.orchestra.run.smithi007.stdout:4268 write 0xd7aaae2 thru 0xd7b3880 (0x8d9f bytes) 2024-09-14T07:00:35.089 INFO:teuthology.orchestra.run.smithi007.stdout:4270 write 0x7bc0eb5 thru 0x7bc9a34 (0x8b80 bytes) 2024-09-14T07:00:35.092 INFO:teuthology.orchestra.run.smithi007.stdout:4271 writesame 0xb227296 thru 0xb228590 data_size 0x1(0x12fb bytes) 2024-09-14T07:00:35.112 INFO:teuthology.orchestra.run.smithi007.stdout:4272 trunc from 0xe2544e7 to 0x420c3f 2024-09-14T07:00:35.145 INFO:teuthology.orchestra.run.smithi007.stdout:4273 writesame 0x5200bc thru 0x523fed data_size 0x1(0x3f32 bytes) 2024-09-14T07:00:35.146 INFO:teuthology.orchestra.run.smithi007.stdout:4274 punch from 0x1cd686 to 0x1d050d, (0x2e87 bytes) 2024-09-14T07:00:35.159 INFO:teuthology.orchestra.run.smithi007.stdout:4275 write 0x6cb39cc thru 0x6cba7fe (0x6e33 bytes) 2024-09-14T07:00:35.162 INFO:teuthology.orchestra.run.smithi007.stdout:4276 write 0x2d09f96 thru 0x2d143ad (0xa418 bytes) 2024-09-14T07:00:35.165 INFO:teuthology.orchestra.run.smithi007.stdout:4279 read 0x2c3eb3 thru 0x2ccf05 (0x9053 bytes) 2024-09-14T07:00:35.165 INFO:teuthology.orchestra.run.smithi007.stdout:4280 write 0x2a261a1 thru 0x2a32450 (0xc2b0 bytes) 2024-09-14T07:00:35.168 INFO:teuthology.orchestra.run.smithi007.stdout:4281 read 0x16cee25 thru 0x16d5762 (0x693e bytes) 2024-09-14T07:00:35.169 INFO:teuthology.orchestra.run.smithi007.stdout:4282 read 0x40d6a2f thru 0x40e3790 (0xcd62 bytes) 2024-09-14T07:00:35.177 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8489 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8208384~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.506051049999996, 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033127139, 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.178 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8489 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670275+0000", 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670276+0000", 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670277+0000", 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6109999999999999e-06 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670283+0000", 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7409999999999998e-06 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670287+0000", 2024-09-14T07:00:35.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1679999999999997e-06 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703311+0000", 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033023640999999999 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703352+0000", 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1168999999999998e-05 2024-09-14T07:00:35.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703402+0000", 2024-09-14T07:00:35.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0424999999999999e-05 2024-09-14T07:00:35.183 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.183 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.183 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.184 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8490 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8323072~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.184 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:35.184 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.506033467000002, 2024-09-14T07:00:35.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033303263, 2024-09-14T07:00:35.184 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.184 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.184 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8490 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670293+0000", 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670295+0000", 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670300+0000", 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5369999999999997e-06 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670304+0000", 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8789999999999996e-06 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703433+0000", 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033128697999999998 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703510+0000", 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6662000000000006e-05 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703596+0000", 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6614999999999997e-05 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.189 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8491 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8454144~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.506015837, 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033389611999999999, 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8491 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.190 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670311+0000", 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4900000000000001e-07 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670317+0000", 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.19e-06 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670323+0000", 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670331+0000", 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1280000000000008e-06 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703611+0000", 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033279557000000001 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703650+0000", 2024-09-14T07:00:35.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9410999999999999e-05 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703700+0000", 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0117000000000001e-05 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8492 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8568832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.194 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505987787000002, 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033561793999999999, 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8492 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670339+0000", 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670341+0000", 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8700000000000001e-06 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670347+0000", 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000004e-06 2024-09-14T07:00:35.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670351+0000", 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4660000000000001e-06 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703731+0000", 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033379502999999998 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703812+0000", 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1192000000000005e-05 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703900+0000", 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8382999999999997e-05 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.199 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8493 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8683520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505969288999999, 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033663516999999997, 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8493 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670357+0000", 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670359+0000", 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.685e-06 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670365+0000", 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.57e-06 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670369+0000", 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4460000000000003e-06 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703928+0000", 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033558530000000003 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.703970+0000", 2024-09-14T07:00:35.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1909999999999997e-05 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704021+0000", 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0988e-05 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8494 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8798208~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.204 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505939693000002, 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033828579999999997, 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8494 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670387+0000", 2024-09-14T07:00:35.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670389+0000", 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.64e-06 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670393+0000", 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3470000000000001e-06 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670396+0000", 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6369999999999999e-06 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704051+0000", 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033655293000000003 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704128+0000", 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7453000000000005e-05 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704215+0000", 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6884000000000004e-05 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8495 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 8912896~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505926143000003, 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033930725000000002, 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.210 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8495 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670400+0000", 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670401+0000", 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.15e-07 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670402+0000", 2024-09-14T07:00:35.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.850000000000001e-07 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670405+0000", 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.225e-06 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670407+0000", 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.34e-06 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704240+0000", 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033833019999999998 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704281+0000", 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1332999999999999e-05 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704331+0000", 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9506999999999999e-05 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8496 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9027584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505914818999997, 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034114617, 2024-09-14T07:00:35.215 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8496 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670412+0000", 2024-09-14T07:00:35.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670413+0000", 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8999999999999995e-07 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670416+0000", 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2729999999999998e-06 2024-09-14T07:00:35.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670418+0000", 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3709999999999998e-06 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704362+0000", 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.033943917999999997 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704439+0000", 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6922999999999995e-05 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704526+0000", 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7035999999999998e-05 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8497 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9142272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505904931000003, 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034220315000000001, 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.221 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8497 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670421+0000", 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670422+0000", 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670426+0000", 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5029999999999999e-06 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670430+0000", 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-06 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670432+0000", 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704552+0000", 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034120277999999997 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704592+0000", 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9884e-05 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704642+0000", 2024-09-14T07:00:35.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9623999999999998e-05 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8498 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9256960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505891167999998, 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034407434000000001, 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.226 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8498 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670435+0000", 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000001e-07 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670436+0000", 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9299999999999996e-07 2024-09-14T07:00:35.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670439+0000", 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670442+0000", 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3120000000000001e-06 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704673+0000", 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034231175000000003 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704749+0000", 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6069999999999995e-05 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704843+0000", 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3622000000000006e-05 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8499 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9371648~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505881647999999, 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034523731000000002, 2024-09-14T07:00:35.231 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8499 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:35.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670445+0000", 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2000000000000001e-07 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670446+0000", 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670449+0000", 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0809999999999998e-06 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670455+0000", 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7930000000000003e-06 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704869+0000", 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034413931000000002 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704910+0000", 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1032999999999998e-05 2024-09-14T07:00:35.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.704968+0000", 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8897999999999998e-05 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8500 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9486336~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.236 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505868370999998, 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034708731, 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8500 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.237 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670458+0000", 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3300000000000001e-07 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670459+0000", 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2999999999999999e-07 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670462+0000", 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1350000000000001e-06 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670470+0000", 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7400000000000004e-06 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705002+0000", 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034531470000000002 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705079+0000", 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7472999999999995e-05 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705167+0000", 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7750000000000005e-05 2024-09-14T07:00:35.241 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8501 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9601024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505849941999998, 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034806278000000003, 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.242 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8501 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670476+0000", 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670477+0000", 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.44e-07 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670479+0000", 2024-09-14T07:00:35.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7740000000000001e-06 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670485+0000", 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9150000000000001e-06 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670489+0000", 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5469999999999998e-06 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705191+0000", 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034701896000000003 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705234+0000", 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2394000000000001e-05 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705283+0000", 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9208e-05 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8502 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9715712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505829603000002, 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034979270999999999, 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.247 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8502 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670497+0000", 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670506+0000", 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9740000000000008e-06 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670512+0000", 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670517+0000", 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7049999999999998e-06 2024-09-14T07:00:35.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705313+0000", 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034795858999999998 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705391+0000", 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8166999999999999e-05 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705476+0000", 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5285999999999996e-05 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8503 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9830400~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505803180999997, 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053094, 2024-09-14T07:00:35.252 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8503 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670523+0000", 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670524+0000", 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670525+0000", 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.581e-06 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670531+0000", 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8969999999999996e-06 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670536+0000", 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4630000000000003e-06 2024-09-14T07:00:35.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705488+0000", 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.034952451000000002 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705526+0000", 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8396999999999998e-05 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705576+0000", 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9926000000000003e-05 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8504 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 9945088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505784843000001, 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035228601999999998, 2024-09-14T07:00:35.257 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8504 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:35.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670542+0000", 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670543+0000", 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5820000000000001e-06 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670549+0000", 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.896e-06 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670554+0000", 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3699999999999997e-06 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705607+0000", 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035053594 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705685+0000", 2024-09-14T07:00:35.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8102999999999994e-05 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705770+0000", 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4678000000000005e-05 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8505 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10059776~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.262 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505767056000003, 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035318548999999998, 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8505 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670559+0000", 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670560+0000", 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670561+0000", 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.494e-06 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670568+0000", 2024-09-14T07:00:35.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9789999999999996e-06 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670572+0000", 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2509999999999996e-06 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705784+0000", 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035211724 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705827+0000", 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3276999999999999e-05 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705878+0000", 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0488000000000002e-05 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8506 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10174464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505746969999997, 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035495512999999999, 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.268 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8506 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670579+0000", 2024-09-14T07:00:35.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670580+0000", 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670582+0000", 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670587+0000", 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.767e-06 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670592+0000", 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2819999999999998e-06 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705909+0000", 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.03531753 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.705987+0000", 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7787999999999995e-05 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706075+0000", 2024-09-14T07:00:35.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7998999999999996e-05 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8507 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10289152~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505729144, 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035582134000000001, 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.273 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8507 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670597+0000", 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670598+0000", 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9900000000000002e-07 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670599+0000", 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6029999999999999e-06 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670609+0000", 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.004e-05 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670614+0000", 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0749999999999997e-06 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706089+0000", 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035474358999999997 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706129+0000", 2024-09-14T07:00:35.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0204999999999999e-05 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706179+0000", 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0553000000000002e-05 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:8508 3.b 3:d0b966a3:::rbd_data.114f7dc3575d.0000000000000007:8 [read 10403840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:35.278 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 51.505705515000002, 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035743559000000001, 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 8508 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670621+0000", 2024-09-14T07:00:35.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2800000000000002e-07 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670623+0000", 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670629+0000", 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.02e-06 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.670634+0000", 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6280000000000004e-06 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706200+0000", 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.035566564000000002 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706276+0000", 2024-09-14T07:00:35.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5801000000000002e-05 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.706364+0000", 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8176000000000004e-05 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:35.283 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:35.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:35.288 INFO:teuthology.orchestra.run.smithi007.stdout:4283 write 0xc0b4126 thru 0xc0bf83a (0xb715 bytes) 2024-09-14T07:00:35.288 INFO:teuthology.orchestra.run.smithi007.stdout:4284 write 0x301fe13 thru 0x3027d23 (0x7f11 bytes) 2024-09-14T07:00:35.288 INFO:teuthology.orchestra.run.smithi007.stdout:4285 read 0x51fa1b3 thru 0x5205d14 (0xbb62 bytes) 2024-09-14T07:00:35.288 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.288 INFO:teuthology.orchestra.run.smithi007.stdout:4289 trunc from 0xc0bf83b to 0x1768a26 2024-09-14T07:00:35.288 INFO:teuthology.orchestra.run.smithi007.stdout:4290 read 0xbbcd13 thru 0xbc7249 (0xa537 bytes) 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:4294 writesame 0x2727b4d thru 0x2735dd1 data_size 0x1(0xe285 bytes) 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:4296 read 0xb00c70 thru 0xb06dd8 (0x6169 bytes) 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:4297 trunc from 0x2735dd2 to 0xad28c6d 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:4304 read 0x3f4bf83 thru 0x3f5013f (0x41bd bytes) 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:4309 punch from 0x2c43ef1 to 0x2c4593d, (0x1a4c bytes) 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:4312 write 0xb1d38ea thru 0xb1e05a9 (0xccc0 bytes) 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:4313 read 0x9ae776d thru 0x9ae9e58 (0x26ec bytes) 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.289 INFO:teuthology.orchestra.run.smithi007.stdout:4315 write 0x1317de2 thru 0x131f25c (0x747b bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4316 read 0x484c3ce thru 0x485a16d (0xdda0 bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4317 write 0x6ba6a81 thru 0x6baa513 (0x3a93 bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4318 write 0x648c60e thru 0x649a38f (0xdd82 bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4320 writesame 0x8bf71f5 thru 0x8c03a3b data_size 0xb(0xc847 bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4322 writesame 0x2f4607f thru 0x2f48b4d data_size 0xd(0x2acf bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4323 writesame 0xe7303b6 thru 0xe73c149 data_size 0x16(0xbd94 bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4325 write 0x6f171f8 thru 0x6f21ee4 (0xaced bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4327 punch from 0x9c5e753 to 0x9c6b6cb, (0xcf78 bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4328 read 0xad93ab thru 0xae19b1 (0x8607 bytes) 2024-09-14T07:00:35.290 INFO:teuthology.orchestra.run.smithi007.stdout:4329 read 0x966a037 thru 0x9671fc6 (0x7f90 bytes) 2024-09-14T07:00:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:4330 read 0x75d5561 thru 0x75e4a1b (0xf4bb bytes) 2024-09-14T07:00:35.291 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:35.283+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:35.291 INFO:teuthology.orchestra.run.smithi007.stdout:4331 trunc from 0xe73c14a to 0x4643b90 2024-09-14T07:00:35.303 INFO:teuthology.orchestra.run.smithi007.stdout:4333 writesame 0x6698921 thru 0x669e592 data_size 0x1(0x5c72 bytes) 2024-09-14T07:00:35.303 INFO:teuthology.orchestra.run.smithi007.stdout:4335 punch from 0xb1d21c to 0xb2c9eb, (0xf7cf bytes) 2024-09-14T07:00:35.303 INFO:teuthology.orchestra.run.smithi007.stdout:4339 read 0x1256c9c thru 0x1257cea (0x104f bytes) 2024-09-14T07:00:35.306 INFO:teuthology.orchestra.run.smithi007.stdout:4340 trunc from 0x669e593 to 0x39e160d 2024-09-14T07:00:35.329 INFO:teuthology.orchestra.run.smithi007.stdout:4341 write 0xdd78189 thru 0xdd7d834 (0x56ac bytes) 2024-09-14T07:00:35.331 INFO:teuthology.orchestra.run.smithi007.stdout:4342 read 0x5c3ca0d thru 0x5c4ac46 (0xe23a bytes) 2024-09-14T07:00:35.336 INFO:teuthology.orchestra.run.smithi007.stdout:4343 trunc from 0xdd7d835 to 0x9ae8a1b 2024-09-14T07:00:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:4344 punch from 0x69ff81f to 0x6a0e0e9, (0xe8ca bytes) 2024-09-14T07:00:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:4345 write 0x8688d1c thru 0x8694416 (0xb6fb bytes) 2024-09-14T07:00:35.348 INFO:teuthology.orchestra.run.smithi007.stdout:4346 write 0x6ae02c8 thru 0x6ae441e (0x4157 bytes) 2024-09-14T07:00:35.351 INFO:teuthology.orchestra.run.smithi007.stdout:4347 read 0x50c7c6c thru 0x50ce1dc (0x6571 bytes) 2024-09-14T07:00:35.352 INFO:teuthology.orchestra.run.smithi007.stdout:4351 writesame 0x22102ff thru 0x2214ebe data_size 0x1(0x4bc0 bytes) 2024-09-14T07:00:35.368 INFO:teuthology.orchestra.run.smithi007.stdout:4352 trunc from 0x9ae8a1b to 0x42c65fb 2024-09-14T07:00:35.378 INFO:teuthology.orchestra.run.smithi007.stdout:4353 write 0x27809dd thru 0x278890a (0x7f2e bytes) 2024-09-14T07:00:35.385 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:35.384+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:35.388 INFO:teuthology.orchestra.run.smithi007.stdout:4354 write 0x6e31155 thru 0x6e32b14 (0x19c0 bytes) 2024-09-14T07:00:35.405 INFO:teuthology.orchestra.run.smithi007.stdout:4355 writesame 0xe2a6871 thru 0xe2abd10 data_size 0x10(0x54a0 bytes) 2024-09-14T07:00:35.405 INFO:teuthology.orchestra.run.smithi007.stdout:4356 read 0x1945958 thru 0x194b780 (0x5e29 bytes) 2024-09-14T07:00:35.405 INFO:teuthology.orchestra.run.smithi007.stdout:4357 read 0xc181709 thru 0xc18d0b0 (0xb9a8 bytes) 2024-09-14T07:00:35.406 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.406 INFO:teuthology.orchestra.run.smithi007.stdout:4359 write 0x2e5a88c thru 0x2e5f12d (0x48a2 bytes) 2024-09-14T07:00:35.412 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:35.412 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:35.412 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:35.413 INFO:teuthology.orchestra.run.smithi007.stdout:4360 trunc from 0xe2abd11 to 0xaa8745e 2024-09-14T07:00:35.418 INFO:teuthology.orchestra.run.smithi007.stdout:4361 read 0xb42e2b thru 0xb448fb (0x1ad1 bytes) 2024-09-14T07:00:35.419 INFO:teuthology.orchestra.run.smithi007.stdout:4362 writesame 0x559016b thru 0x559849c data_size 0x7(0x8332 bytes) 2024-09-14T07:00:35.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:35.430 INFO:teuthology.orchestra.run.smithi007.stdout:4363 write 0xbd1b761 thru 0xbd26521 (0xadc1 bytes) 2024-09-14T07:00:35.435 INFO:teuthology.orchestra.run.smithi007.stdout:4365 trunc from 0xbd26522 to 0xdd61e87 2024-09-14T07:00:35.441 INFO:teuthology.orchestra.run.smithi007.stdout:4366 read 0xac314aa thru 0xac3cc8f (0xb7e6 bytes) 2024-09-14T07:00:35.441 INFO:teuthology.orchestra.run.smithi007.stdout:4368 write 0x6ef86e9 thru 0x6effde4 (0x76fc bytes) 2024-09-14T07:00:35.445 INFO:teuthology.orchestra.run.smithi007.stdout:4370 write 0x5b5caa3 thru 0x5b6b3f9 (0xe957 bytes) 2024-09-14T07:00:35.449 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.449 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.449 INFO:teuthology.orchestra.run.smithi007.stdout:4377 read 0x6b3d618 thru 0x6b48993 (0xb37c bytes) 2024-09-14T07:00:35.449 INFO:teuthology.orchestra.run.smithi007.stdout:4378 write 0x5e51722 thru 0x5e53a8d (0x236c bytes) 2024-09-14T07:00:35.452 INFO:teuthology.orchestra.run.smithi007.stdout:4379 write 0x3e5ca5f thru 0x3e6705a (0xa5fc bytes) 2024-09-14T07:00:35.455 INFO:teuthology.orchestra.run.smithi007.stdout:4380 write 0x580cbc6 thru 0x5819354 (0xc78f bytes) 2024-09-14T07:00:35.458 INFO:teuthology.orchestra.run.smithi007.stdout:4381 write 0xb1167b9 thru 0xb126534 (0xfd7c bytes) 2024-09-14T07:00:35.461 INFO:teuthology.orchestra.run.smithi007.stdout:4382 read 0x4ea7361 thru 0x4eac6c2 (0x5362 bytes) 2024-09-14T07:00:35.462 INFO:teuthology.orchestra.run.smithi007.stdout:4383 read 0x3b8f889 thru 0x3b9da78 (0xe1f0 bytes) 2024-09-14T07:00:35.467 INFO:teuthology.orchestra.run.smithi007.stdout:4384 writesame 0xe1d08d6 thru 0xe1d5673 data_size 0x5(0x4d9e bytes) 2024-09-14T07:00:35.467 INFO:teuthology.orchestra.run.smithi007.stdout:4385 writesame 0x893efd thru 0x8a1cb4 data_size 0x58(0xddb8 bytes) 2024-09-14T07:00:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:4387 read 0xc55932a thru 0xc55a98a (0x1661 bytes) 2024-09-14T07:00:35.469 INFO:teuthology.orchestra.run.smithi007.stdout:4390 punch from 0x7f351c7 to 0x7f35d6a, (0xba3 bytes) 2024-09-14T07:00:35.481 INFO:teuthology.orchestra.run.smithi007.stdout:4391 trunc from 0xe1d5674 to 0x2c59cf7 2024-09-14T07:00:35.485 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:35.484+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:35.493 INFO:teuthology.orchestra.run.smithi007.stdout:4392 read 0x64fbfb thru 0x6560cb (0x64d1 bytes) 2024-09-14T07:00:35.494 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.494 INFO:teuthology.orchestra.run.smithi007.stdout:4394 punch from 0x2f4624 to 0x2f7634, (0x3010 bytes) 2024-09-14T07:00:35.494 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.494 INFO:teuthology.orchestra.run.smithi007.stdout:4397 write 0x25a1672 thru 0x25a5a9e (0x442d bytes) 2024-09-14T07:00:35.505 INFO:teuthology.orchestra.run.smithi007.stdout:4398 writesame 0x5624a02 thru 0x5630b87 data_size 0x17(0xc186 bytes) 2024-09-14T07:00:35.516 INFO:teuthology.orchestra.run.smithi007.stdout:4399 trunc from 0x5630b88 to 0xed890a3 2024-09-14T07:00:35.530 INFO:teuthology.orchestra.run.smithi007.stdout:4400 trunc from 0xed890a3 to 0x701c128 2024-09-14T07:00:35.538 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.539 INFO:teuthology.orchestra.run.smithi007.stdout:4402 punch from 0x417070c to 0x41712d0, (0xbc4 bytes) 2024-09-14T07:00:35.539 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:35.539 INFO:teuthology.orchestra.run.smithi007.stdout:4404 read 0x1d49f91 thru 0x1d54e10 (0xae80 bytes) 2024-09-14T07:00:35.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:00:35.549 INFO:teuthology.orchestra.run.smithi007.stdout:4405 write 0xaa8f3df thru 0xaa9f253 (0xfe75 bytes) 2024-09-14T07:00:35.554 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:35.554 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:35.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:35.586 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:35.586+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:35.687 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:35.685+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:35.716 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:35.716 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:35.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:35.716 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:35.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.716 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.716 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:35.716 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.310364268999997, 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:00:35.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.310302428999996, 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:00:35.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.726 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.310269253000001, 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:00:35.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.310146381999999, 2024-09-14T07:00:35.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:00:35.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:35.736 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.310111632999998, 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:00:35.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.310087705000001, 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:00:35.741 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:35.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:00:35.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.310065471000001, 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:00:35.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.310032503000002, 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:00:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:00:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.309681732000001, 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:00:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.309577548999997, 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:00:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.309537038999999, 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:35.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:00:35.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:00:35.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.309511919000002, 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:35.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:00:35.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.309357622, 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:00:35.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:00:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:00:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:00:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:00:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.786 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.309324625999999, 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:00:35.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:00:35.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.309301720999997, 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:00:35.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:00:35.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:00:35.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.309251717999999, 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:00:35.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.114519666, 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:35.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:00:35.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:00:35.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.109813152000001, 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:00:35.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:00:35.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.106429487, 2024-09-14T07:00:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:00:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:00:35.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:00:35.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.106409752000001, 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:35.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:35.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:00:35.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:35.822 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:35.823 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:35.824 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:35.822+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:35.924 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:35.922+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:35.972 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:35.972 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:35.972 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:35.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:36.025 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:36.024+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:36.125 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:36.125+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:36.126 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:36.127 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:36.127 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:36.127 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:36.127 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:36.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:36.227 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:36.226+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:36.244 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:00:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.508715477999999, 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:00:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:00:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:00:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17618 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.006102540000001, 2024-09-14T07:00:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016649290000000001, 2024-09-14T07:00:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17618 2024-09-14T07:00:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236332+0000", 2024-09-14T07:00:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-14T07:00:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237113+0000", 2024-09-14T07:00:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078082900000000003 2024-09-14T07:00:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237131+0000", 2024-09-14T07:00:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7747000000000001e-05 2024-09-14T07:00:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237141+0000", 2024-09-14T07:00:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0137e-05 2024-09-14T07:00:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237221+0000", 2024-09-14T07:00:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0482000000000006e-05 2024-09-14T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237383+0000", 2024-09-14T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016176100000000001 2024-09-14T07:00:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237737+0000", 2024-09-14T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035380999999999998 2024-09-14T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.241638+0000", 2024-09-14T07:00:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039011739999999999 2024-09-14T07:00:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252854+0000", 2024-09-14T07:00:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011216097 2024-09-14T07:00:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252919+0000", 2024-09-14T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5129999999999995e-05 2024-09-14T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252979+0000", 2024-09-14T07:00:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9573e-05 2024-09-14T07:00:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40284 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:9 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.037975479000004, 2024-09-14T07:00:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016005640000000002, 2024-09-14T07:00:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40284 2024-09-14T07:00:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204457+0000", 2024-09-14T07:00:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:00:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204459+0000", 2024-09-14T07:00:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204466+0000", 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8730000000000001e-06 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204472+0000", 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4779999999999999e-06 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207358+0000", 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028862549999999999 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207430+0000", 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2268000000000007e-05 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220462+0000", 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013031951999999999 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40646 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.036833844999997, 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017879991000000001, 2024-09-14T07:00:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40646 2024-09-14T07:00:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:00:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205605+0000", 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2260000000000002e-06 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205611+0000", 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3289999999999999e-06 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205622+0000", 2024-09-14T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-14T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213726+0000", 2024-09-14T07:00:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008104745 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213792+0000", 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5401999999999995e-05 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223478+0000", 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096862100000000007 2024-09-14T07:00:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40683 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.036761925999997, 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016494009, 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40683 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205672+0000", 2024-09-14T07:00:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0379999999999998e-06 2024-09-14T07:00:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205679+0000", 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205684+0000", 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9359999999999998e-06 2024-09-14T07:00:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221845+0000", 2024-09-14T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016161422000000002 2024-09-14T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221913+0000", 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7669000000000006e-05 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222164+0000", 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025130600000000002 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40685 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.036741522, 2024-09-14T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016657634000000001, 2024-09-14T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40685 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205691+0000", 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205693+0000", 2024-09-14T07:00:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-14T07:00:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205699+0000", 2024-09-14T07:00:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205703+0000", 2024-09-14T07:00:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5240000000000002e-06 2024-09-14T07:00:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222190+0000", 2024-09-14T07:00:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016486752 2024-09-14T07:00:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222257+0000", 2024-09-14T07:00:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7095999999999996e-05 2024-09-14T07:00:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222348+0000", 2024-09-14T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1061000000000001e-05 2024-09-14T07:00:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40687 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.036722292, 2024-09-14T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016826543999999999, 2024-09-14T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40687 2024-09-14T07:00:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4999999999999998e-07 2024-09-14T07:00:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205712+0000", 2024-09-14T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205718+0000", 2024-09-14T07:00:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:00:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205723+0000", 2024-09-14T07:00:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222374+0000", 2024-09-14T07:00:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016650933999999999 2024-09-14T07:00:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222445+0000", 2024-09-14T07:00:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1304999999999996e-05 2024-09-14T07:00:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222536+0000", 2024-09-14T07:00:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1104000000000001e-05 2024-09-14T07:00:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40720 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.036605696000002, 2024-09-14T07:00:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016893371000000001, 2024-09-14T07:00:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40720 2024-09-14T07:00:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205827+0000", 2024-09-14T07:00:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.58e-07 2024-09-14T07:00:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205834+0000", 2024-09-14T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2729999999999997e-06 2024-09-14T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205841+0000", 2024-09-14T07:00:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:00:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205846+0000", 2024-09-14T07:00:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1769999999999997e-06 2024-09-14T07:00:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222562+0000", 2024-09-14T07:00:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016716178000000002 2024-09-14T07:00:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222633+0000", 2024-09-14T07:00:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0461000000000002e-05 2024-09-14T07:00:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222720+0000", 2024-09-14T07:00:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6997999999999996e-05 2024-09-14T07:00:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40721 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.036577993999998, 2024-09-14T07:00:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044822000000001, 2024-09-14T07:00:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40721 2024-09-14T07:00:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205857+0000", 2024-09-14T07:00:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-06 2024-09-14T07:00:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205863+0000", 2024-09-14T07:00:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-14T07:00:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205868+0000", 2024-09-14T07:00:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T07:00:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222745+0000", 2024-09-14T07:00:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016877293000000002 2024-09-14T07:00:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222814+0000", 2024-09-14T07:00:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8806000000000005e-05 2024-09-14T07:00:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222899+0000", 2024-09-14T07:00:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4913000000000005e-05 2024-09-14T07:00:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40742 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.036557428000002, 2024-09-14T07:00:36.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017201659000000001, 2024-09-14T07:00:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40742 2024-09-14T07:00:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:36.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-14T07:00:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205892+0000", 2024-09-14T07:00:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7008e-05 2024-09-14T07:00:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205899+0000", 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1489999999999997e-06 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205904+0000", 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222923+0000", 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017018229999999999 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222991+0000", 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8545000000000002e-05 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223076+0000", 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4938999999999994e-05 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40743 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.036518696000002, 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343654999999999, 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40743 2024-09-14T07:00:36.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205914+0000", 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-07 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205916+0000", 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0559999999999999e-06 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205922+0000", 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3659999999999997e-06 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205927+0000", 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8169999999999998e-06 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223101+0000", 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017174452 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223168+0000", 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6756999999999998e-05 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223257+0000", 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8700000000000001e-05 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43579 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.786570067, 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01522916, 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43579 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455864+0000", 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:36.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455870+0000", 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455875+0000", 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9509999999999997e-06 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470879+0000", 2024-09-14T07:00:36.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015003729 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470960+0000", 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1228000000000002e-05 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471091+0000", 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131018 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43580 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 110592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.786550529000003, 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015447047, 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43580 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.344 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455882+0000", 2024-09-14T07:00:36.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455884+0000", 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2010000000000002e-06 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455891+0000", 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5690000000000001e-06 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455896+0000", 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3639999999999998e-06 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471123+0000", 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015226805 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471200+0000", 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7405999999999997e-05 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471328+0000", 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012835099999999999 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43581 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.786528627999999, 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015668949000000001, 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43581 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455904+0000", 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455910+0000", 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6619999999999999e-06 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455917+0000", 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5769999999999999e-06 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455923+0000", 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6779999999999997e-06 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471360+0000", 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015436854999999999 2024-09-14T07:00:36.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471443+0000", 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3815999999999998e-05 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471572+0000", 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012892900000000001 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117541 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.350826692999998, 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015320307999999999, 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117541 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891606+0000", 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891609+0000", 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.886e-06 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891615+0000", 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5010000000000001e-06 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891620+0000", 2024-09-14T07:00:36.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901891+0000", 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010270833 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901942+0000", 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0612000000000003e-05 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:36.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902142+0000", 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020015500000000001 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904269+0000", 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021265519999999999 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906874+0000", 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026050800000000001 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906904+0000", 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199e-05 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906926+0000", 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1708e-05 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117542 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:36.358 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.350804994000001, 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015409499, 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117542 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891630+0000", 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8569999999999999e-06 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891638+0000", 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5610000000000001e-06 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891643+0000", 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4299999999999997e-06 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902805+0000", 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011162071000000001 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902879+0000", 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4152000000000004e-05 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903129+0000", 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024969499999999997 2024-09-14T07:00:36.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904277+0000", 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001147906 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906968+0000", 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002690942 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906999+0000", 2024-09-14T07:00:36.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1056000000000002e-05 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.907036+0000", 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7410000000000003e-05 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117543 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.350772534000001, 2024-09-14T07:00:36.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018002384999999999, 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117543 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891660+0000", 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8100000000000003e-07 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891663+0000", 2024-09-14T07:00:36.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2739999999999999e-06 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891671+0000", 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.303e-06 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891678+0000", 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3709999999999996e-06 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903501+0000", 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011823382 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903577+0000", 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5947999999999999e-05 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903855+0000", 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027763899999999999 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904798+0000", 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094325599999999996 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909603+0000", 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048047539999999996 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909634+0000", 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1137999999999999e-05 2024-09-14T07:00:36.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909662+0000", 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7838999999999999e-05 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:120396 3.1c 3:3ebb3b40:::rbd_object_map.114f74d20483.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.683807817999998, 2024-09-14T07:00:36.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015664009999999999, 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 120396 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558625+0000", 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558631+0000", 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558647+0000", 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5756999999999998e-05 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558657+0000", 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.995e-06 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558724+0000", 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7008000000000006e-05 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558815+0000", 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1077999999999998e-05 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.559173+0000", 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035768600000000002 2024-09-14T07:00:36.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560127+0000", 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095468799999999998 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574168+0000", 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014040238999999999 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:36.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574233+0000", 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999994e-05 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574288+0000", 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.558e-05 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:134959 3.1a 3:59279a6c:::rbd_data.114fdc2d0201.0000000000000072:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:00:36.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:21.970619+0000", 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.271813164999999, 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021043874000000001, 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 134959 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970619+0000", 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970619+0000", 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970620+0000", 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970625+0000", 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5450000000000003e-06 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970638+0000", 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2777e-05 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970646+0000", 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9680000000000006e-06 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970714+0000", 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8361999999999994e-05 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970796+0000", 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1577999999999997e-05 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.974492+0000", 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036963090000000001 2024-09-14T07:00:36.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991537+0000", 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044678000000001 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991586+0000", 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8689000000000001e-05 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991627+0000", 2024-09-14T07:00:36.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1085999999999999e-05 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991663+0000", 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5936999999999998e-05 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.384 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138654 3.1c 3:3ed33fa1:::rbd_data.114f74bd3bbf.00000000000000cc:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-14T07:00:36.384 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:31.531314+0000", 2024-09-14T07:00:36.384 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7111177319999999, 2024-09-14T07:00:36.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016015504, 2024-09-14T07:00:36.384 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:36.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138654 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531314+0000", 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531314+0000", 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531315+0000", 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000004e-07 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531319+0000", 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1999999999999996e-06 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531330+0000", 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0414999999999999e-05 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531336+0000", 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2389999999999999e-06 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531382+0000", 2024-09-14T07:00:36.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5867e-05 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531474+0000", 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2489000000000002e-05 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.534588+0000", 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031140849999999999 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:36.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547207+0000", 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012618994999999999 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547261+0000", 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3269000000000001e-05 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547298+0000", 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7471000000000001e-05 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:36.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547330+0000", 2024-09-14T07:00:36.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1680000000000002e-05 2024-09-14T07:00:36.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:36.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:36.390 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:36.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:36.392 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:36.390+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:36.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:36.492 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:36.492+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:36.545 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T07:00:36.547 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:36.547 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:36.547 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:36.550 INFO:teuthology.orchestra.run.smithi007.stdout:4407 write 0x67af8b thru 0x68a04e (0xf0c4 bytes) 2024-09-14T07:00:36.555 INFO:teuthology.orchestra.run.smithi007.stdout:4408 write 0x91e5647 thru 0x91e8a0c (0x33c6 bytes) 2024-09-14T07:00:36.559 INFO:teuthology.orchestra.run.smithi007.stdout:4409 writesame 0x42e1b64 thru 0x42e74a3 data_size 0x70(0x5940 bytes) 2024-09-14T07:00:36.562 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:36.563 INFO:teuthology.orchestra.run.smithi007.stdout:4411 read 0x3b1fc16 thru 0x3b236d5 (0x3ac0 bytes) 2024-09-14T07:00:36.563 INFO:teuthology.orchestra.run.smithi007.stdout:4412 punch from 0x40dde2 to 0x41a176, (0xc394 bytes) 2024-09-14T07:00:36.565 INFO:teuthology.orchestra.run.smithi007.stdout:4413 trunc from 0xaa9f254 to 0x886f11d 2024-09-14T07:00:36.589 INFO:teuthology.orchestra.run.smithi007.stdout:4415 writesame 0xd0c5e00 thru 0xd0c61cb data_size 0x12(0x3cc bytes) 2024-09-14T07:00:36.589 INFO:teuthology.orchestra.run.smithi007.stdout:4417 punch from 0xa3cc90f to 0xa3d10e6, (0x47d7 bytes) 2024-09-14T07:00:36.589 INFO:teuthology.orchestra.run.smithi007.stdout:4418 punch from 0x39ecf30 to 0x39f99aa, (0xca7a bytes) 2024-09-14T07:00:36.589 INFO:teuthology.orchestra.run.smithi007.stdout:4419 write 0xca083c6 thru 0xca10602 (0x823d bytes) 2024-09-14T07:00:36.593 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:36.593+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:36.599 INFO:teuthology.orchestra.run.smithi007.stdout:4421 write 0xd9aa86a thru 0xd9ada57 (0x31ee bytes) 2024-09-14T07:00:36.602 INFO:teuthology.orchestra.run.smithi007.stdout:4424 read 0x1a31bcb thru 0x1a3d5ad (0xb9e3 bytes) 2024-09-14T07:00:36.602 INFO:teuthology.orchestra.run.smithi007.stdout:4425 punch from 0x1cb0e16 to 0x1cbf057, (0xe241 bytes) 2024-09-14T07:00:36.602 INFO:teuthology.orchestra.run.smithi007.stdout:4426 read 0x4de4a20 thru 0x4defe7a (0xb45b bytes) 2024-09-14T07:00:36.611 INFO:teuthology.orchestra.run.smithi007.stdout:4427 trunc from 0xd9ada58 to 0x60f3d86 2024-09-14T07:00:36.622 INFO:teuthology.orchestra.run.smithi007.stdout:4428 read 0x3d31cc1 thru 0x3d32c6b (0xfab bytes) 2024-09-14T07:00:36.628 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:36.628 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:36.634 INFO:teuthology.orchestra.run.smithi007.stdout:4429 writesame 0xbc9c190 thru 0xbca5b40 data_size 0x5(0x99b1 bytes) 2024-09-14T07:00:36.634 INFO:teuthology.orchestra.run.smithi007.stdout:4430 punch from 0x75e8948 to 0x75f455a, (0xbc12 bytes) 2024-09-14T07:00:36.634 INFO:teuthology.orchestra.run.smithi007.stdout:4431 read 0x18b1f53 thru 0x18b8a7a (0x6b28 bytes) 2024-09-14T07:00:36.635 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:36.635 INFO:teuthology.orchestra.run.smithi007.stdout:4434 read 0x15b0170 thru 0x15b5077 (0x4f08 bytes) 2024-09-14T07:00:36.635 INFO:teuthology.orchestra.run.smithi007.stdout:4438 read 0xb161d48 thru 0xb1691b1 (0x746a bytes) 2024-09-14T07:00:36.635 INFO:teuthology.orchestra.run.smithi007.stdout:4440 punch from 0x90494e3 to 0x9057be0, (0xe6fd bytes) 2024-09-14T07:00:36.641 INFO:teuthology.orchestra.run.smithi007.stdout:4441 trunc from 0xbca5b41 to 0x6f554a7 2024-09-14T07:00:36.647 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:36.654 INFO:teuthology.orchestra.run.smithi007.stdout:4442 trunc from 0x6f554a7 to 0xc51f388 2024-09-14T07:00:36.659 INFO:teuthology.orchestra.run.smithi007.stdout:4443 punch from 0x2d9e2a0 to 0x2da77db, (0x953b bytes) 2024-09-14T07:00:36.665 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:36.667 INFO:teuthology.orchestra.run.smithi007.stdout:4444 trunc from 0xc51f388 to 0x598f837 2024-09-14T07:00:36.673 INFO:teuthology.orchestra.run.smithi007.stdout:4445 read 0x22ea80b thru 0x22f1367 (0x6b5d bytes) 2024-09-14T07:00:36.683 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:36.684 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:36.686 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:36.686 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:36.687 INFO:teuthology.orchestra.run.smithi007.stdout:4446 write 0xd591182 thru 0xd5955f1 (0x4470 bytes) 2024-09-14T07:00:36.691 INFO:teuthology.orchestra.run.smithi007.stdout:4448 writesame 0xbe0a3b5 thru 0xbe14fc1 data_size 0x17(0xac0d bytes) 2024-09-14T07:00:36.694 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:36.693+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:36.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:36.701 INFO:teuthology.orchestra.run.smithi007.stdout:4451 write 0xe32489a thru 0xe325b1d (0x1284 bytes) 2024-09-14T07:00:36.701 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:36.702 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:36.704 INFO:teuthology.orchestra.run.smithi007.stdout:4452 read 0x46d7afa thru 0x46ddb2e (0x6035 bytes) 2024-09-14T07:00:36.704 INFO:teuthology.orchestra.run.smithi007.stdout:4453 read 0xc131af3 thru 0xc141928 (0xfe36 bytes) 2024-09-14T07:00:36.704 INFO:teuthology.orchestra.run.smithi007.stdout:4456 writesame 0x3d1142f thru 0x3d1b401 data_size 0x31(0x9fd3 bytes) 2024-09-14T07:00:36.707 INFO:teuthology.orchestra.run.smithi007.stdout:4457 write 0x23cad91 thru 0x23d53f5 (0xa665 bytes) 2024-09-14T07:00:36.720 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:36.721 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:36.721 INFO:teuthology.orchestra.run.smithi007.stdout:4458 trunc from 0xe325b1e to 0x5870ba7 2024-09-14T07:00:36.735 INFO:teuthology.orchestra.run.smithi007.stdout:4459 read 0x4e45b33 thru 0x4e4814c (0x261a bytes) 2024-09-14T07:00:36.735 INFO:teuthology.orchestra.run.smithi007.stdout:4460 read 0x312c6e8 thru 0x31348eb (0x8204 bytes) 2024-09-14T07:00:36.735 INFO:teuthology.orchestra.run.smithi007.stdout:4461 read 0x1113fc9 thru 0x111fa01 (0xba39 bytes) 2024-09-14T07:00:36.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:36.741 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:36.742 INFO:teuthology.orchestra.run.smithi007.stdout:4464 trunc from 0x5870ba7 to 0xb3173bd 2024-09-14T07:00:36.747 INFO:teuthology.orchestra.run.smithi007.stdout:4466 write 0x6480f99 thru 0x6484e9a (0x3f02 bytes) 2024-09-14T07:00:36.751 INFO:teuthology.orchestra.run.smithi007.stdout:4467 read 0x72b2811 thru 0x72bac8d (0x847d bytes) 2024-09-14T07:00:36.751 INFO:teuthology.orchestra.run.smithi007.stdout:4468 read 0x8ecf805 thru 0x8ed8e7b (0x9677 bytes) 2024-09-14T07:00:36.751 INFO:teuthology.orchestra.run.smithi007.stdout:4469 punch from 0x4792573 to 0x479b6e0, (0x916d bytes) 2024-09-14T07:00:36.751 INFO:teuthology.orchestra.run.smithi007.stdout:4471 punch from 0x541f5c9 to 0x542abfa, (0xb631 bytes) 2024-09-14T07:00:36.751 INFO:teuthology.orchestra.run.smithi007.stdout:4472 write 0x2a42178 thru 0x2a50e72 (0xecfb bytes) 2024-09-14T07:00:36.755 INFO:teuthology.orchestra.run.smithi007.stdout:4473 write 0x9f3e56d thru 0x9f47552 (0x8fe6 bytes) 2024-09-14T07:00:36.759 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:36.759 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:36.769 INFO:teuthology.orchestra.run.smithi007.stdout:4475 writesame 0xd26ea1a thru 0xd279688 data_size 0x1(0xac6f bytes) 2024-09-14T07:00:36.769 INFO:teuthology.orchestra.run.smithi007.stdout:4476 punch from 0xbdcbb85 to 0xbdd1202, (0x567d bytes) 2024-09-14T07:00:36.769 INFO:teuthology.orchestra.run.smithi007.stdout:4479 read 0xd010a8b thru 0xd01600d (0x5583 bytes) 2024-09-14T07:00:36.769 INFO:teuthology.orchestra.run.smithi007.stdout:4481 punch from 0x52bc6de to 0x52cc253, (0xfb75 bytes) 2024-09-14T07:00:36.775 INFO:teuthology.orchestra.run.smithi007.stdout:4485 write 0xdb36079 thru 0xdb3f65d (0x95e5 bytes) 2024-09-14T07:00:36.778 INFO:teuthology.orchestra.run.smithi007.stdout:4486 write 0x1186227 thru 0x1189b77 (0x3951 bytes) 2024-09-14T07:00:36.780 INFO:teuthology.orchestra.run.smithi007.stdout:4487 punch from 0x9ddd71c to 0x9de2c12, (0x54f6 bytes) 2024-09-14T07:00:36.780 INFO:teuthology.orchestra.run.smithi007.stdout:4488 clone 28 order 19 su 262144 sc 6 2024-09-14T07:00:36.794 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:36.793+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:36.819 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:36.819 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:36.828 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:36.895 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:36.894+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:36.980 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:36.980 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:36.989 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:37.036 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:37.034+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:37.097 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:37.096+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:37.098 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:37.098 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:37.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:37.198 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:37.196+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:37.255 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:37.255 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:37.263 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:37.305 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:37.303+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:37.369 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:37.370 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:37.378 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:37.422 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:37.421+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:37.482 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:37.482 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:37.490 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:37.532 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:37.531+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:37.543 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:37.544 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:37.544 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-14T07:00:37.544 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:00:37.544 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:00:37.544 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-14T07:00:37.609 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone27 from 0xdb3f65e (overlap 0x420c3f) to 0x1a6b4c 2024-09-14T07:00:37.619 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:37.618+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:37.650 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:37.650 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:37.659 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:37.680 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-14T07:00:37.691 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:37.692 INFO:teuthology.orchestra.run.smithi189.stdout: "success": "" 2024-09-14T07:00:37.692 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:37.743 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:37.742+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:37.800 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:37.800 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:37.803 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:37.803+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:37.808 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:37.904 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:37.902+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:37.925 INFO:teuthology.orchestra.run.smithi007.stdout:4490 write 0xe121f77 thru 0xe12cd3a (0xadc4 bytes) 2024-09-14T07:00:37.928 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:37.928 INFO:teuthology.orchestra.run.smithi007.stdout:4492 write 0x82b66d2 thru 0x82be501 (0x7e30 bytes) 2024-09-14T07:00:37.932 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:37.932 INFO:teuthology.orchestra.run.smithi007.stdout:4497 read 0x7a5c4b2 thru 0x7a622e6 (0x5e35 bytes) 2024-09-14T07:00:37.938 INFO:teuthology.orchestra.run.smithi007.stdout:4498 write 0xe6ffedf thru 0xe702563 (0x2685 bytes) 2024-09-14T07:00:37.942 INFO:teuthology.orchestra.run.smithi007.stdout:4501 writesame 0x16564f0 thru 0x165b338 data_size 0x7(0x4e49 bytes) 2024-09-14T07:00:37.946 INFO:teuthology.orchestra.run.smithi007.stdout:4502 writesame 0x8831ee5 thru 0x883608e data_size 0xa(0x41aa bytes) 2024-09-14T07:00:37.949 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:37.957 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:37.958 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:37.964 INFO:teuthology.orchestra.run.smithi007.stdout:4505 trunc from 0xe702564 to 0xe9646c 2024-09-14T07:00:37.967 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:37.994 INFO:teuthology.orchestra.run.smithi007.stdout:4506 trunc from 0xe9646c to 0xd671838 2024-09-14T07:00:38.001 INFO:teuthology.orchestra.run.smithi007.stdout:4507 write 0xce475cc thru 0xce4df92 (0x69c7 bytes) 2024-09-14T07:00:38.004 INFO:teuthology.orchestra.run.smithi007.stdout:4508 punch from 0xa62da56 to 0xa62ef22, (0x14cc bytes) 2024-09-14T07:00:38.004 INFO:teuthology.orchestra.run.smithi007.stdout:4510 writesame 0xcb62c9b thru 0xcb65b96 data_size 0x4(0x2efc bytes) 2024-09-14T07:00:38.004 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:38.003+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:38.008 INFO:teuthology.orchestra.run.smithi007.stdout:4511 read 0x5befa6b thru 0x5bf240d (0x29a3 bytes) 2024-09-14T07:00:38.008 INFO:teuthology.orchestra.run.smithi007.stdout:4512 read 0x8d4c1b8 thru 0x8d54ae3 (0x892c bytes) 2024-09-14T07:00:38.008 INFO:teuthology.orchestra.run.smithi007.stdout:4514 punch from 0x5894994 to 0x58a24ae, (0xdb1a bytes) 2024-09-14T07:00:38.008 INFO:teuthology.orchestra.run.smithi007.stdout:4515 writesame 0x59c5f4a thru 0x59c87de data_size 0x1(0x2895 bytes) 2024-09-14T07:00:38.014 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.014 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.014 INFO:teuthology.orchestra.run.smithi007.stdout:4519 read 0x9e85035 thru 0x9e87c74 (0x2c40 bytes) 2024-09-14T07:00:38.014 INFO:teuthology.orchestra.run.smithi007.stdout:4520 read 0x82e5d33 thru 0x82ee829 (0x8af7 bytes) 2024-09-14T07:00:38.015 INFO:teuthology.orchestra.run.smithi007.stdout:4523 write 0x64b810b thru 0x64c12d5 (0x91cb bytes) 2024-09-14T07:00:38.020 INFO:teuthology.orchestra.run.smithi007.stdout:4524 read 0x5dc497b thru 0x5dcd181 (0x8807 bytes) 2024-09-14T07:00:38.021 INFO:teuthology.orchestra.run.smithi007.stdout:4525 write 0x439f13b thru 0x43a445f (0x5325 bytes) 2024-09-14T07:00:38.030 INFO:teuthology.orchestra.run.smithi007.stdout:4526 write 0xda304fc thru 0xda3fdec (0xf8f1 bytes) 2024-09-14T07:00:38.033 INFO:teuthology.orchestra.run.smithi007.stdout:4527 read 0x9a6f981 thru 0x9a7e19b (0xe81b bytes) 2024-09-14T07:00:38.048 INFO:teuthology.orchestra.run.smithi007.stdout:4529 trunc from 0xda3fded to 0x5fa299 2024-09-14T07:00:38.064 INFO:teuthology.orchestra.run.smithi007.stdout:4530 read 0x3c3adb thru 0x3c5349 (0x186f bytes) 2024-09-14T07:00:38.064 INFO:teuthology.orchestra.run.smithi007.stdout:4531 clone 29 order 23 su 1048576 sc 8 2024-09-14T07:00:38.078 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:38.079 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:38.087 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:38.132 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:38.131+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:38.191 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:38.191 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:38.199 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:38.206 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:38.205+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:38.307 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:38.306+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:38.342 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:38.343 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:38.351 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:38.408 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:38.406+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:38.494 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:38.494 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:38.504 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:38.548 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:38.547+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:38.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:00:38.612 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:38.610+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:38.612 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:38.612 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:38.621 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:38.623 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone28 from 0x5fa299 (overlap 0x5fa299) to 0x561256 2024-09-14T07:00:38.688 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-14T07:00:38.693 INFO:teuthology.orchestra.run.smithi007.stdout:4532 read 0x157f7b thru 0x15d45b (0x54e1 bytes) 2024-09-14T07:00:38.694 INFO:teuthology.orchestra.run.smithi007.stdout:4535 punch from 0x688ec to 0x73ad9, (0xb1ed bytes) 2024-09-14T07:00:38.697 INFO:teuthology.orchestra.run.smithi007.stdout:4536 punch from 0x3267ac to 0x331347, (0xab9b bytes) 2024-09-14T07:00:38.710 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:38.710+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:38.718 INFO:teuthology.orchestra.run.smithi007.stdout:4537 write 0xec29462 thru 0xec2af09 (0x1aa8 bytes) 2024-09-14T07:00:38.721 INFO:teuthology.orchestra.run.smithi007.stdout:4538 write 0x158a6a thru 0x15a862 (0x1df9 bytes) 2024-09-14T07:00:38.728 INFO:teuthology.orchestra.run.smithi007.stdout:4539 punch from 0x7bf8d4b to 0x7c0804a, (0xf2ff bytes) 2024-09-14T07:00:38.729 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.729 INFO:teuthology.orchestra.run.smithi007.stdout:4541 punch from 0xe01b841 to 0xe023979, (0x8138 bytes) 2024-09-14T07:00:38.729 INFO:teuthology.orchestra.run.smithi007.stdout:4544 read 0x44b59c8 thru 0x44c4eee (0xf527 bytes) 2024-09-14T07:00:38.729 INFO:teuthology.orchestra.run.smithi007.stdout:4546 punch from 0x142e85b to 0x1432b86, (0x432b bytes) 2024-09-14T07:00:38.729 INFO:teuthology.orchestra.run.smithi007.stdout:4547 read 0x1a7ca6a thru 0x1a86870 (0x9e07 bytes) 2024-09-14T07:00:38.729 INFO:teuthology.orchestra.run.smithi007.stdout:4548 read 0xe01d522 thru 0xe021e70 (0x494f bytes) 2024-09-14T07:00:38.729 INFO:teuthology.orchestra.run.smithi007.stdout:4551 read 0xdae9759 thru 0xdaf1cab (0x8553 bytes) 2024-09-14T07:00:38.729 INFO:teuthology.orchestra.run.smithi007.stdout:4552 writesame 0x5622dee thru 0x562bcb1 data_size 0x2(0x8ec4 bytes) 2024-09-14T07:00:38.738 INFO:teuthology.orchestra.run.smithi007.stdout:4554 write 0xe9c3537 thru 0xe9c3fee (0xab8 bytes) 2024-09-14T07:00:38.742 INFO:teuthology.orchestra.run.smithi007.stdout:4555 read 0xb1439a7 thru 0xb1491e0 (0x583a bytes) 2024-09-14T07:00:38.742 INFO:teuthology.orchestra.run.smithi007.stdout:4557 read 0x2875e30 thru 0x2876de9 (0xfba bytes) 2024-09-14T07:00:38.742 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.742 INFO:teuthology.orchestra.run.smithi007.stdout:4559 writesame 0x79a2e83 thru 0x79b23b4 data_size 0x5(0xf532 bytes) 2024-09-14T07:00:38.752 INFO:teuthology.orchestra.run.smithi007.stdout:4561 read 0xab6b436 thru 0xab7010f (0x4cda bytes) 2024-09-14T07:00:38.752 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.752 INFO:teuthology.orchestra.run.smithi007.stdout:4563 write 0x510e404 thru 0x511b39e (0xcf9b bytes) 2024-09-14T07:00:38.754 INFO:teuthology.orchestra.run.smithi007.stdout:4564 punch from 0x543f998 to 0x5449fb1, (0xa619 bytes) 2024-09-14T07:00:38.754 INFO:teuthology.orchestra.run.smithi007.stdout:4566 read 0x9925d2b thru 0x9934c80 (0xef56 bytes) 2024-09-14T07:00:38.754 INFO:teuthology.orchestra.run.smithi007.stdout:4567 writesame 0x40d3f9e thru 0x40e112d data_size 0x38(0xd190 bytes) 2024-09-14T07:00:38.757 INFO:teuthology.orchestra.run.smithi007.stdout:4568 write 0xcd79130 thru 0xcd7bb77 (0x2a48 bytes) 2024-09-14T07:00:38.761 INFO:teuthology.orchestra.run.smithi007.stdout:4570 punch from 0x36f9b54 to 0x3708166, (0xe612 bytes) 2024-09-14T07:00:38.761 INFO:teuthology.orchestra.run.smithi007.stdout:4572 read 0xbc55e36 thru 0xbc6320a (0xd3d5 bytes) 2024-09-14T07:00:38.761 INFO:teuthology.orchestra.run.smithi007.stdout:4574 read 0x4dc5cf3 thru 0x4dca55d (0x486b bytes) 2024-09-14T07:00:38.761 INFO:teuthology.orchestra.run.smithi007.stdout:4576 read 0xc7a3698 thru 0xc7ae05c (0xa9c5 bytes) 2024-09-14T07:00:38.761 INFO:teuthology.orchestra.run.smithi007.stdout:4577 read 0x3214c86 thru 0x3215111 (0x48c bytes) 2024-09-14T07:00:38.761 INFO:teuthology.orchestra.run.smithi007.stdout:4579 read 0x6313a7d thru 0x631b063 (0x75e7 bytes) 2024-09-14T07:00:38.761 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.761 INFO:teuthology.orchestra.run.smithi007.stdout:4582 trunc from 0xec2af0a to 0xdb280e2 2024-09-14T07:00:38.769 INFO:teuthology.orchestra.run.smithi007.stdout:4583 punch from 0x5e0a1ad to 0x5e1373d, (0x9590 bytes) 2024-09-14T07:00:38.769 INFO:teuthology.orchestra.run.smithi007.stdout:4585 write 0x628aab4 thru 0x62978c0 (0xce0d bytes) 2024-09-14T07:00:38.771 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:38.771 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:38.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:38.772 INFO:teuthology.orchestra.run.smithi007.stdout:4586 read 0x7c3643a thru 0x7c38b4a (0x2711 bytes) 2024-09-14T07:00:38.772 INFO:teuthology.orchestra.run.smithi007.stdout:4587 writesame 0x5c717ec thru 0x5c799eb data_size 0x82(0x8200 bytes) 2024-09-14T07:00:38.775 INFO:teuthology.orchestra.run.smithi007.stdout:4589 read 0x3c5cc27 thru 0x3c5e680 (0x1a5a bytes) 2024-09-14T07:00:38.776 INFO:teuthology.orchestra.run.smithi007.stdout:4591 flatten 2024-09-14T07:00:38.780 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:38.793 INFO:teuthology.orchestra.run.smithi007.stdout:4592 read 0xd84c692 thru 0xd858712 (0xc081 bytes) 2024-09-14T07:00:38.793 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.793 INFO:teuthology.orchestra.run.smithi007.stdout:4595 write 0xb8944fd thru 0xb895dee (0x18f2 bytes) 2024-09-14T07:00:38.796 INFO:teuthology.orchestra.run.smithi007.stdout:4597 read 0x8f485d8 thru 0x8f4f9e8 (0x7411 bytes) 2024-09-14T07:00:38.797 INFO:teuthology.orchestra.run.smithi007.stdout:4598 read 0x686e77e thru 0x686f9cb (0x124e bytes) 2024-09-14T07:00:38.797 INFO:teuthology.orchestra.run.smithi007.stdout:4599 write 0x765afc7 thru 0x7669a60 (0xea9a bytes) 2024-09-14T07:00:38.798 INFO:teuthology.orchestra.run.smithi007.stdout:4600 write 0x21e9637 thru 0x21f5bde (0xc5a8 bytes) 2024-09-14T07:00:38.801 INFO:teuthology.orchestra.run.smithi007.stdout:4601 trunc from 0xdb280e2 to 0xc6572da 2024-09-14T07:00:38.809 INFO:teuthology.orchestra.run.smithi007.stdout:4602 write 0x4059df3 thru 0x4061af0 (0x7cfe bytes) 2024-09-14T07:00:38.810 INFO:teuthology.orchestra.run.smithi007.stdout:4603 read 0x3b63338 thru 0x3b6c129 (0x8df2 bytes) 2024-09-14T07:00:38.820 INFO:teuthology.orchestra.run.smithi007.stdout:4604 trunc from 0xc6572da to 0x4a81da2 2024-09-14T07:00:38.822 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:38.821+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:38.842 INFO:teuthology.orchestra.run.smithi007.stdout:4605 trunc from 0x4a81da2 to 0xd138542 2024-09-14T07:00:38.848 INFO:teuthology.orchestra.run.smithi007.stdout:4608 punch from 0x71da194 to 0x71dc2e2, (0x214e bytes) 2024-09-14T07:00:38.848 INFO:teuthology.orchestra.run.smithi007.stdout:4610 writesame 0x9fa0051 thru 0x9fa7311 data_size 0x1(0x72c1 bytes) 2024-09-14T07:00:38.878 INFO:teuthology.orchestra.run.smithi007.stdout:4611 trunc from 0xd138542 to 0x124e88 2024-09-14T07:00:38.893 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.907 INFO:teuthology.orchestra.run.smithi007.stdout:4613 write 0x6c15167 thru 0x6c2420a (0xf0a4 bytes) 2024-09-14T07:00:38.911 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.911 INFO:teuthology.orchestra.run.smithi007.stdout:4615 write 0x6749a34 thru 0x6752425 (0x89f2 bytes) 2024-09-14T07:00:38.911 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:38.911+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:38.914 INFO:teuthology.orchestra.run.smithi007.stdout:4616 read 0x67a04e6 thru 0x67a5ef8 (0x5a13 bytes) 2024-09-14T07:00:38.914 INFO:teuthology.orchestra.run.smithi007.stdout:4619 read 0x6202a14 thru 0x6210a4a (0xe037 bytes) 2024-09-14T07:00:38.915 INFO:teuthology.orchestra.run.smithi007.stdout:4622 writesame 0x29332a4 thru 0x2940c5a data_size 0x5(0xd9b7 bytes) 2024-09-14T07:00:38.921 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:38.921 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:38.921 INFO:teuthology.orchestra.run.smithi007.stdout:4623 write 0xffe540 thru 0x100d7b9 (0xf27a bytes) 2024-09-14T07:00:38.931 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:38.931 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:38.934 INFO:teuthology.orchestra.run.smithi007.stdout:4624 writesame 0xb20248b thru 0xb20f635 data_size 0x5(0xd1ab bytes) 2024-09-14T07:00:38.935 INFO:teuthology.orchestra.run.smithi007.stdout:4625 writesame 0x6279bab thru 0x62846e2 data_size 0x4(0xab38 bytes) 2024-09-14T07:00:38.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:38.941 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:38.941 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:38.943 INFO:teuthology.orchestra.run.smithi007.stdout:4626 write 0x9bc788b thru 0x9bcf6fe (0x7e74 bytes) 2024-09-14T07:00:38.946 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.946 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.946 INFO:teuthology.orchestra.run.smithi007.stdout:4631 read 0x9d3efdb thru 0x9d42504 (0x352a bytes) 2024-09-14T07:00:38.946 INFO:teuthology.orchestra.run.smithi007.stdout:4632 read 0x369c92a thru 0x36a924d (0xc924 bytes) 2024-09-14T07:00:38.946 INFO:teuthology.orchestra.run.smithi007.stdout:4633 read 0x83e82c1 thru 0x83f22be (0x9ffe bytes) 2024-09-14T07:00:38.946 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.947 INFO:teuthology.orchestra.run.smithi007.stdout:4635 read 0xa406591 thru 0xa40b8c3 (0x5333 bytes) 2024-09-14T07:00:38.947 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.947 INFO:teuthology.orchestra.run.smithi007.stdout:4638 punch from 0xa1ec114 to 0xa1f5f87, (0x9e73 bytes) 2024-09-14T07:00:38.954 INFO:teuthology.orchestra.run.smithi007.stdout:4639 writesame 0xc72ffcf thru 0xc73f025 data_size 0x1(0xf057 bytes) 2024-09-14T07:00:38.954 INFO:teuthology.orchestra.run.smithi007.stdout:4642 write 0xa4d5fd4 thru 0xa4e5041 (0xf06e bytes) 2024-09-14T07:00:38.957 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:38.962 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:38.966 INFO:teuthology.orchestra.run.smithi007.stdout:4646 write 0xebe2e94 thru 0xebe4832 (0x199f bytes) 2024-09-14T07:00:38.969 INFO:teuthology.orchestra.run.smithi007.stdout:4647 read 0x4011fd6 thru 0x40211e5 (0xf210 bytes) 2024-09-14T07:00:38.969 INFO:teuthology.orchestra.run.smithi007.stdout:4648 writesame 0xa18a81e thru 0xa18e6c9 data_size 0x2a(0x3eac bytes) 2024-09-14T07:00:38.981 INFO:teuthology.orchestra.run.smithi007.stdout:4649 trunc from 0xebe4833 to 0x79ab557 2024-09-14T07:00:38.987 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:38.987 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:38.994 INFO:teuthology.orchestra.run.smithi007.stdout:4650 punch from 0x44fa86c to 0x4507f00, (0xd694 bytes) 2024-09-14T07:00:38.994 INFO:teuthology.orchestra.run.smithi007.stdout:4651 punch from 0x1d3cd96 to 0x1d440f9, (0x7363 bytes) 2024-09-14T07:00:38.994 INFO:teuthology.orchestra.run.smithi007.stdout:4652 read 0x58d7867 thru 0x58db511 (0x3cab bytes) 2024-09-14T07:00:39.003 INFO:teuthology.orchestra.run.smithi007.stdout:4653 writesame 0xb35b14b thru 0xb365419 data_size 0x21(0xa2cf bytes) 2024-09-14T07:00:39.004 INFO:teuthology.orchestra.run.smithi007.stdout:4655 write 0x1f196ab thru 0x1f1fed7 (0x682d bytes) 2024-09-14T07:00:39.005 INFO:teuthology.orchestra.run.smithi007.stdout:4656 write 0x6549d68 thru 0x6552037 (0x82d0 bytes) 2024-09-14T07:00:39.006 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:39.007 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:39.012 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:39.011+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:39.021 INFO:teuthology.orchestra.run.smithi007.stdout:4657 trunc from 0xb36541a to 0x1c3f23d 2024-09-14T07:00:39.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:39.035 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:39.035 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.036 INFO:teuthology.orchestra.run.smithi007.stdout:4660 read 0x828bba thru 0x837a5d (0xeea4 bytes) 2024-09-14T07:00:39.053 INFO:teuthology.orchestra.run.smithi007.stdout:4661 write 0x91afc1a thru 0x91bf5df (0xf9c6 bytes) 2024-09-14T07:00:39.055 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.055 INFO:teuthology.orchestra.run.smithi007.stdout:4667 write 0x64e69d6 thru 0x64ef44e (0x8a79 bytes) 2024-09-14T07:00:39.059 INFO:teuthology.orchestra.run.smithi007.stdout:4670 read 0x6d4980 thru 0x6e1b45 (0xd1c6 bytes) 2024-09-14T07:00:39.061 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:39.061 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:39.067 INFO:teuthology.orchestra.run.smithi007.stdout:4672 writesame 0xb6cdcc8 thru 0xb6d55e5 data_size 0x25(0x791e bytes) 2024-09-14T07:00:39.067 INFO:teuthology.orchestra.run.smithi007.stdout:4673 read 0x4e892e7 thru 0x4e973d1 (0xe0eb bytes) 2024-09-14T07:00:39.068 INFO:teuthology.orchestra.run.smithi007.stdout:4674 read 0x7773af4 thru 0x777ed91 (0xb29e bytes) 2024-09-14T07:00:39.068 INFO:teuthology.orchestra.run.smithi007.stdout:4677 write 0x481f039 thru 0x482e382 (0xf34a bytes) 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:141742 3.3 3:c1f4d278:::rbd_object_map.114fb2a419f1:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:39.068024+0000", 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001677607, 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017001220000000001, 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:39.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 141742 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.068024+0000", 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.068024+0000", 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.068025+0000", 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100000000000001e-06 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.068030+0000", 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.068040+0000", 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0325e-05 2024-09-14T07:00:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.068046+0000", 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2430000000000002e-06 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.068103+0000", 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7068000000000003e-05 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.068199+0000", 2024-09-14T07:00:39.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5173000000000005e-05 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.068616+0000", 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00041697099999999998 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069458+0000", 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00084287399999999997 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069637+0000", 2024-09-14T07:00:39.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017871900000000001 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069693+0000", 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069737+0000", 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4134999999999997e-05 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4431.0:141742 3.3 e85/76 ondisk, result = 0)", 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:39.069548+0000", 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.00015353900000000001, 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023828, 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069548+0000", 2024-09-14T07:00:39.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069548+0000", 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069549+0000", 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3900000000000003e-07 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069553+0000", 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2649999999999998e-06 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069565+0000", 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1997999999999999e-05 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069572+0000", 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5300000000000002e-06 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069622+0000", 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0188000000000001e-05 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:39.069631+0000", 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2520000000000005e-06 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:39.079 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 2 2024-09-14T07:00:39.080 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:39.080 INFO:teuthology.orchestra.run.smithi007.stdout:4678 punch from 0x46736ee to 0x467b414, (0x7d26 bytes) 2024-09-14T07:00:39.080 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.082 INFO:teuthology.orchestra.run.smithi007.stdout:4680 trunc from 0xb6d55e6 to 0x1961c1d 2024-09-14T07:00:39.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:39.089 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:39.090 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:39.102 INFO:teuthology.orchestra.run.smithi007.stdout:4682 writesame 0x3813df0 thru 0x3823ad3 data_size 0x53(0xfce4 bytes) 2024-09-14T07:00:39.103 INFO:teuthology.orchestra.run.smithi007.stdout:4684 read 0x8ff26d thru 0x8ffefd (0xc91 bytes) 2024-09-14T07:00:39.113 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:39.112+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:39.119 INFO:teuthology.orchestra.run.smithi007.stdout:4685 write 0xd39dc65 thru 0xd3ac45d (0xe7f9 bytes) 2024-09-14T07:00:39.123 INFO:teuthology.orchestra.run.smithi007.stdout:4686 read 0x4e79228 thru 0x4e7fa21 (0x67fa bytes) 2024-09-14T07:00:39.123 INFO:teuthology.orchestra.run.smithi007.stdout:4687 punch from 0x3d22736 to 0x3d23127, (0x9f1 bytes) 2024-09-14T07:00:39.123 INFO:teuthology.orchestra.run.smithi007.stdout:4688 read 0x5226f5d thru 0x5234c99 (0xdd3d bytes) 2024-09-14T07:00:39.123 INFO:teuthology.orchestra.run.smithi007.stdout:4689 write 0x2da52b9 thru 0x2dad9a8 (0x86f0 bytes) 2024-09-14T07:00:39.126 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.126 INFO:teuthology.orchestra.run.smithi007.stdout:4691 write 0x39f2226 thru 0x39fb559 (0x9334 bytes) 2024-09-14T07:00:39.128 INFO:teuthology.orchestra.run.smithi007.stdout:4692 trunc from 0xd3ac45e to 0xbfb2125 2024-09-14T07:00:39.144 INFO:teuthology.orchestra.run.smithi007.stdout:4694 writesame 0xce13d95 thru 0xce1beea data_size 0x6e(0x8156 bytes) 2024-09-14T07:00:39.144 INFO:teuthology.orchestra.run.smithi007.stdout:4695 punch from 0x8a07821 to 0x8a1223c, (0xaa1b bytes) 2024-09-14T07:00:39.145 INFO:teuthology.orchestra.run.smithi007.stdout:4696 write 0xb5dd7bf thru 0xb5e7c90 (0xa4d2 bytes) 2024-09-14T07:00:39.147 INFO:teuthology.orchestra.run.smithi007.stdout:4697 write 0xacaa46a thru 0xacb0447 (0x5fde bytes) 2024-09-14T07:00:39.150 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.150 INFO:teuthology.orchestra.run.smithi007.stdout:4701 writesame 0x9a03ef0 thru 0x9a093d8 data_size 0x1(0x54e9 bytes) 2024-09-14T07:00:39.163 INFO:teuthology.orchestra.run.smithi007.stdout:4702 trunc from 0xce1beeb to 0xc9bce45 2024-09-14T07:00:39.168 INFO:teuthology.orchestra.run.smithi007.stdout:4703 write 0x6b796a1 thru 0x6b7b3ea (0x1d4a bytes) 2024-09-14T07:00:39.172 INFO:teuthology.orchestra.run.smithi007.stdout:4704 read 0x910bf29 thru 0x9114622 (0x86fa bytes) 2024-09-14T07:00:39.172 INFO:teuthology.orchestra.run.smithi007.stdout:4705 write 0xc3e1fea thru 0xc3ec8d7 (0xa8ee bytes) 2024-09-14T07:00:39.175 INFO:teuthology.orchestra.run.smithi007.stdout:4706 writesame 0xa242785 thru 0xa24a0ba data_size 0x1d(0x7936 bytes) 2024-09-14T07:00:39.176 INFO:teuthology.orchestra.run.smithi007.stdout:4707 writesame 0xa803e50 thru 0xa80a653 data_size 0x1c(0x6804 bytes) 2024-09-14T07:00:39.179 INFO:teuthology.orchestra.run.smithi007.stdout:4710 read 0x1db550a thru 0x1dbe9b6 (0x94ad bytes) 2024-09-14T07:00:39.180 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.193 INFO:teuthology.orchestra.run.smithi007.stdout:4712 trunc from 0xc9bce45 to 0xdaf189 2024-09-14T07:00:39.205 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.205 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.214 INFO:teuthology.orchestra.run.smithi007.stdout:4715 write 0x3c674bd thru 0x3c72e90 (0xb9d4 bytes) 2024-09-14T07:00:39.214 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:39.213+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:39.217 INFO:teuthology.orchestra.run.smithi007.stdout:4716 write 0x1ccbde4 thru 0x1cd14f8 (0x5715 bytes) 2024-09-14T07:00:39.218 INFO:teuthology.orchestra.run.smithi007.stdout:4717 read 0x12c828e thru 0x12d1db9 (0x9b2c bytes) 2024-09-14T07:00:39.218 INFO:teuthology.orchestra.run.smithi007.stdout:4718 write 0x382c91f thru 0x3837196 (0xa878 bytes) 2024-09-14T07:00:39.219 INFO:teuthology.orchestra.run.smithi007.stdout:4719 read 0xb8feb7 thru 0xb93798 (0x38e2 bytes) 2024-09-14T07:00:39.224 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:39.224 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:39.232 INFO:teuthology.orchestra.run.smithi007.stdout:4720 write 0x94d5af1 thru 0x94db764 (0x5c74 bytes) 2024-09-14T07:00:39.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:39.236 INFO:teuthology.orchestra.run.smithi007.stdout:4722 punch from 0x5409f4c to 0x54180d7, (0xe18b bytes) 2024-09-14T07:00:39.236 INFO:teuthology.orchestra.run.smithi007.stdout:4723 punch from 0x8e19827 to 0x8e22fb1, (0x978a bytes) 2024-09-14T07:00:39.236 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.236 INFO:teuthology.orchestra.run.smithi007.stdout:4725 write 0x383c271 thru 0x3845142 (0x8ed2 bytes) 2024-09-14T07:00:39.237 INFO:teuthology.orchestra.run.smithi007.stdout:4726 write 0x30ffc7c thru 0x3106b79 (0x6efe bytes) 2024-09-14T07:00:39.244 INFO:teuthology.orchestra.run.smithi007.stdout:4727 trunc from 0x94db765 to 0xe7def4c 2024-09-14T07:00:39.252 INFO:teuthology.orchestra.run.smithi007.stdout:4729 read 0x73773e3 thru 0x738091e (0x953c bytes) 2024-09-14T07:00:39.252 INFO:teuthology.orchestra.run.smithi007.stdout:4730 write 0x7ebefa6 thru 0x7ece579 (0xf5d4 bytes) 2024-09-14T07:00:39.253 INFO:teuthology.orchestra.run.smithi007.stdout:4731 read 0x6a194e0 thru 0x6a2298a (0x94ab bytes) 2024-09-14T07:00:39.261 INFO:teuthology.orchestra.run.smithi007.stdout:4734 trunc from 0xe7def4c to 0x6f02f86 2024-09-14T07:00:39.271 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.271 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.271 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.276 INFO:teuthology.orchestra.run.smithi007.stdout:4739 trunc from 0x6f02f86 to 0xb601571 2024-09-14T07:00:39.287 INFO:teuthology.orchestra.run.smithi007.stdout:4740 write 0xbbfe78f thru 0xbc06c3f (0x84b1 bytes) 2024-09-14T07:00:39.291 INFO:teuthology.orchestra.run.smithi007.stdout:4741 read 0x60687c3 thru 0x6068d5a (0x598 bytes) 2024-09-14T07:00:39.300 INFO:teuthology.orchestra.run.smithi007.stdout:4742 writesame 0xe38e63a thru 0xe38e65c data_size 0x1(0x23 bytes) 2024-09-14T07:00:39.300 INFO:teuthology.orchestra.run.smithi007.stdout:4744 read 0xc697654 thru 0xc6a6d25 (0xf6d2 bytes) 2024-09-14T07:00:39.300 INFO:teuthology.orchestra.run.smithi007.stdout:4745 writesame 0x6f302d8 thru 0x6f340fa data_size 0x1(0x3e23 bytes) 2024-09-14T07:00:39.309 INFO:teuthology.orchestra.run.smithi007.stdout:4746 read 0x5825fcc thru 0x582de4a (0x7e7f bytes) 2024-09-14T07:00:39.310 INFO:teuthology.orchestra.run.smithi007.stdout:4747 read 0xa9c46c3 thru 0xa9c5571 (0xeaf bytes) 2024-09-14T07:00:39.315 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:39.313+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:39.325 INFO:teuthology.orchestra.run.smithi007.stdout:4750 trunc from 0xe38e65d to 0xc8e104 2024-09-14T07:00:39.336 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.336 INFO:teuthology.orchestra.run.smithi007.stdout:4753 punch from 0x2e22d to 0x2f6bf, (0x1492 bytes) 2024-09-14T07:00:39.350 INFO:teuthology.orchestra.run.smithi007.stdout:4755 write 0x8de17a9 thru 0x8df01ec (0xea44 bytes) 2024-09-14T07:00:39.363 INFO:teuthology.orchestra.run.smithi007.stdout:4756 write 0xcc9988d thru 0xcca5127 (0xb89b bytes) 2024-09-14T07:00:39.365 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.960198972999997, 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:39.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:00:39.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.960137133000003, 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.960103957000001, 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:00:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:00:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959981085999999, 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959946336999998, 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:00:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:00:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:00:39.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.393 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.393 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:39.393 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959922409000001, 2024-09-14T07:00:39.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:00:39.393 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:39.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959900175000001, 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:00:39.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:00:39.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959867207000002, 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:00:39.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:00:39.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959516436000001, 2024-09-14T07:00:39.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:39.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:00:39.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959412252999996, 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:00:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959371742999998, 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:00:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959346623000002, 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:00:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959192326, 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:39.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959159329999999, 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:00:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:00:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959136424999997, 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:00:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.959086421999999, 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:00:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.76435437, 2024-09-14T07:00:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:00:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:00:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.759647856000001, 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:00:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.756264191, 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:00:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.756244456000001, 2024-09-14T07:00:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.463 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:39.464 INFO:teuthology.orchestra.run.smithi007.stdout:4757 trunc from 0xcca5128 to 0x3d176e2 2024-09-14T07:00:39.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:39.466 INFO:teuthology.orchestra.run.smithi007.stdout:4758 read 0xe7721e thru 0xe7cc13 (0x59f6 bytes) 2024-09-14T07:00:39.466 INFO:teuthology.orchestra.run.smithi007.stdout:4760 read 0x10ef40b thru 0x10fc72a (0xd320 bytes) 2024-09-14T07:00:39.466 INFO:teuthology.orchestra.run.smithi007.stdout:4761 write 0xece9a91 thru 0xecf1536 (0x7aa6 bytes) 2024-09-14T07:00:39.466 INFO:teuthology.orchestra.run.smithi007.stdout:4762 write 0xa2bedce thru 0xa2c089a (0x1acd bytes) 2024-09-14T07:00:39.466 INFO:teuthology.orchestra.run.smithi007.stdout:4763 punch from 0x5858efa to 0x586642b, (0xd531 bytes) 2024-09-14T07:00:39.466 INFO:teuthology.orchestra.run.smithi007.stdout:4764 read 0x1abcb thru 0x2a755 (0xfb8b bytes) 2024-09-14T07:00:39.467 INFO:teuthology.orchestra.run.smithi007.stdout:4765 read 0x4c80b66 thru 0x4c8196e (0xe09 bytes) 2024-09-14T07:00:39.467 INFO:teuthology.orchestra.run.smithi007.stdout:4766 write 0xf508b3 thru 0xf5420a (0x3958 bytes) 2024-09-14T07:00:39.467 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.467 INFO:teuthology.orchestra.run.smithi007.stdout:4768 trunc from 0xecf1537 to 0x5e449c5 2024-09-14T07:00:39.467 INFO:teuthology.orchestra.run.smithi007.stdout:4769 write 0xa12fee6 thru 0xa13002a (0x145 bytes) 2024-09-14T07:00:39.467 INFO:teuthology.orchestra.run.smithi007.stdout:4772 punch from 0x7351ce to 0x73b5b5, (0x63e7 bytes) 2024-09-14T07:00:39.467 INFO:teuthology.orchestra.run.smithi007.stdout:4773 writesame 0x6146ad1 thru 0x6150e4e data_size 0x11(0xa37e bytes) 2024-09-14T07:00:39.467 INFO:teuthology.orchestra.run.smithi007.stdout:4774 write 0xbc8d072 thru 0xbc8d1c5 (0x154 bytes) 2024-09-14T07:00:39.467 INFO:teuthology.orchestra.run.smithi007.stdout:4775 write 0xbf15b99 thru 0xbf17a98 (0x1f00 bytes) 2024-09-14T07:00:39.467 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:39.462+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:39.468 INFO:teuthology.orchestra.run.smithi007.stdout:4776 punch from 0x6bc5530 to 0x6bcf725, (0xa1f5 bytes) 2024-09-14T07:00:39.472 INFO:teuthology.orchestra.run.smithi007.stdout:4777 write 0xd8004da thru 0xd80ac69 (0xa790 bytes) 2024-09-14T07:00:39.475 INFO:teuthology.orchestra.run.smithi007.stdout:4778 writesame 0xc8dbad4 thru 0xc8e873d data_size 0x5(0xcc6a bytes) 2024-09-14T07:00:39.481 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.481 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.481 INFO:teuthology.orchestra.run.smithi007.stdout:4783 write 0xd156a3a thru 0xd1658e0 (0xeea7 bytes) 2024-09-14T07:00:39.484 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.484 INFO:teuthology.orchestra.run.smithi007.stdout:4785 punch from 0x49486f1 to 0x494b983, (0x3292 bytes) 2024-09-14T07:00:39.484 INFO:teuthology.orchestra.run.smithi007.stdout:4786 punch from 0x8c628c6 to 0x8c64967, (0x20a1 bytes) 2024-09-14T07:00:39.492 INFO:teuthology.orchestra.run.smithi007.stdout:4787 trunc from 0xd80ac6a to 0x6cc4a1e 2024-09-14T07:00:39.503 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.515 INFO:teuthology.orchestra.run.smithi007.stdout:4790 writesame 0xc15bd5a thru 0xc166637 data_size 0x83(0xa8de bytes) 2024-09-14T07:00:39.515 INFO:teuthology.orchestra.run.smithi007.stdout:4792 writesame 0x12de397 thru 0x12e173b data_size 0x27(0x33a5 bytes) 2024-09-14T07:00:39.518 INFO:teuthology.orchestra.run.smithi007.stdout:4793 read 0x1ec85e9 thru 0x1ed2ccf (0xa6e7 bytes) 2024-09-14T07:00:39.530 INFO:teuthology.orchestra.run.smithi007.stdout:4795 trunc from 0xc166638 to 0x187f40c 2024-09-14T07:00:39.541 INFO:teuthology.orchestra.run.smithi007.stdout:4796 read 0xa9502c thru 0xaa427f (0xf254 bytes) 2024-09-14T07:00:39.546 INFO:teuthology.orchestra.run.smithi007.stdout:4797 trunc from 0x187f40c to 0x5b61372 2024-09-14T07:00:39.553 INFO:teuthology.orchestra.run.smithi007.stdout:4798 write 0x2d5420d thru 0x2d62841 (0xe635 bytes) 2024-09-14T07:00:39.555 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T07:00:39.564 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:39.564+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:39.572 INFO:teuthology.orchestra.run.smithi007.stdout:4799 write 0xada1333 thru 0xada6eb3 (0x5b81 bytes) 2024-09-14T07:00:39.584 INFO:teuthology.orchestra.run.smithi007.stdout:4800 trunc from 0xada6eb4 to 0x3e04ac8 2024-09-14T07:00:39.612 INFO:teuthology.orchestra.run.smithi007.stdout:4802 write 0xd561981 thru 0xd57008b (0xe70b bytes) 2024-09-14T07:00:39.614 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:39.614 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:39.614 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:39.614 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:39.615 INFO:teuthology.orchestra.run.smithi007.stdout:4803 punch from 0x3221e3e to 0x322680f, (0x49d1 bytes) 2024-09-14T07:00:39.615 INFO:teuthology.orchestra.run.smithi007.stdout:4805 writesame 0x3152d16 thru 0x31568cc data_size 0x1(0x3bb7 bytes) 2024-09-14T07:00:39.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:39.624 INFO:teuthology.orchestra.run.smithi007.stdout:4807 read 0x8bf25ec thru 0x8c00653 (0xe068 bytes) 2024-09-14T07:00:39.624 INFO:teuthology.orchestra.run.smithi007.stdout:4808 read 0x530a984 thru 0x530ccf2 (0x236f bytes) 2024-09-14T07:00:39.624 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.624 INFO:teuthology.orchestra.run.smithi007.stdout:4810 write 0x95b5c72 thru 0x95bc8ec (0x6c7b bytes) 2024-09-14T07:00:39.627 INFO:teuthology.orchestra.run.smithi007.stdout:4811 read 0xb79fcbf thru 0xb7ad95c (0xdc9e bytes) 2024-09-14T07:00:39.627 INFO:teuthology.orchestra.run.smithi007.stdout:4812 writesame 0x4d1f56c thru 0x4d2daed data_size 0x1d(0xe582 bytes) 2024-09-14T07:00:39.631 INFO:teuthology.orchestra.run.smithi007.stdout:4813 read 0x790b137 thru 0x790e92e (0x37f8 bytes) 2024-09-14T07:00:39.631 INFO:teuthology.orchestra.run.smithi007.stdout:4814 read 0xab93748 thru 0xab96c56 (0x350f bytes) 2024-09-14T07:00:39.631 INFO:teuthology.orchestra.run.smithi007.stdout:4815 writesame 0xc22c9dd thru 0xc235174 data_size 0x4(0x8798 bytes) 2024-09-14T07:00:39.637 INFO:teuthology.orchestra.run.smithi007.stdout:4816 trunc from 0xd57008c to 0xdfa9a26 2024-09-14T07:00:39.642 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.643 INFO:teuthology.orchestra.run.smithi007.stdout:4819 write 0x613356f thru 0x6136aaa (0x353c bytes) 2024-09-14T07:00:39.659 INFO:teuthology.orchestra.run.smithi007.stdout:4820 trunc from 0xdfa9a26 to 0x27fccfa 2024-09-14T07:00:39.666 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:39.665+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:39.687 INFO:teuthology.orchestra.run.smithi007.stdout:4822 writesame 0xbebcb80 thru 0xbec26c8 data_size 0x1(0x5b49 bytes) 2024-09-14T07:00:39.687 INFO:teuthology.orchestra.run.smithi007.stdout:4823 write 0x3f7407 thru 0x3fa246 (0x2e40 bytes) 2024-09-14T07:00:39.690 INFO:teuthology.orchestra.run.smithi007.stdout:4824 write 0x9a1bb05 thru 0x9a2b943 (0xfe3f bytes) 2024-09-14T07:00:39.694 INFO:teuthology.orchestra.run.smithi007.stdout:4825 punch from 0x9dd221c to 0x9dd8f1c, (0x6d00 bytes) 2024-09-14T07:00:39.694 INFO:teuthology.orchestra.run.smithi007.stdout:4827 punch from 0x8369069 to 0x8376141, (0xd0d8 bytes) 2024-09-14T07:00:39.694 INFO:teuthology.orchestra.run.smithi007.stdout:4828 read 0xbb2e34 thru 0xbb7e9c (0x5069 bytes) 2024-09-14T07:00:39.694 INFO:teuthology.orchestra.run.smithi007.stdout:4829 read 0x45c160e thru 0x45c7d2a (0x671d bytes) 2024-09-14T07:00:39.694 INFO:teuthology.orchestra.run.smithi007.stdout:4830 writesame 0x98eb1ee thru 0x98eef41 data_size 0x32(0x3d54 bytes) 2024-09-14T07:00:39.697 INFO:teuthology.orchestra.run.smithi007.stdout:4832 punch from 0x84124ae to 0x8416169, (0x3cbb bytes) 2024-09-14T07:00:39.698 INFO:teuthology.orchestra.run.smithi007.stdout:4833 punch from 0x1f6e43 to 0x2028d1, (0xba8e bytes) 2024-09-14T07:00:39.707 INFO:teuthology.orchestra.run.smithi007.stdout:4837 trunc from 0xbec26c9 to 0x37faa81 2024-09-14T07:00:39.720 INFO:teuthology.orchestra.run.smithi007.stdout:4839 trunc from 0x37faa81 to 0x18ff647 2024-09-14T07:00:39.727 INFO:teuthology.orchestra.run.smithi007.stdout:4841 punch from 0x14dd091 to 0x14e935d, (0xc2cc bytes) 2024-09-14T07:00:39.727 INFO:teuthology.orchestra.run.smithi007.stdout:4843 read 0x1190419 thru 0x119257d (0x2165 bytes) 2024-09-14T07:00:39.728 INFO:teuthology.orchestra.run.smithi007.stdout:4844 read 0xac9f95 thru 0xad2850 (0x88bc bytes) 2024-09-14T07:00:39.733 INFO:teuthology.orchestra.run.smithi007.stdout:4845 write 0x1bc045d thru 0x1bc44db (0x407f bytes) 2024-09-14T07:00:39.740 INFO:teuthology.orchestra.run.smithi007.stdout:4846 write 0x1fc0d3b thru 0x1fca92c (0x9bf2 bytes) 2024-09-14T07:00:39.751 INFO:teuthology.orchestra.run.smithi007.stdout:4849 write 0x530189e thru 0x530e070 (0xc7d3 bytes) 2024-09-14T07:00:39.752 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:39.752 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:39.753 INFO:teuthology.orchestra.run.smithi007.stdout:4851 read 0x516ea5b thru 0x517818b (0x9731 bytes) 2024-09-14T07:00:39.766 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:39.765+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:39.770 INFO:teuthology.orchestra.run.smithi007.stdout:4853 write 0xd7b659c thru 0xd7bf031 (0x8a96 bytes) 2024-09-14T07:00:39.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:39.773 INFO:teuthology.orchestra.run.smithi007.stdout:4854 write 0x141d1c4 thru 0x142b654 (0xe491 bytes) 2024-09-14T07:00:39.776 INFO:teuthology.orchestra.run.smithi007.stdout:4855 read 0x9eba8f4 thru 0x9ec67b6 (0xbec3 bytes) 2024-09-14T07:00:39.776 INFO:teuthology.orchestra.run.smithi007.stdout:4857 writesame 0xbbf1eca thru 0xbbfd64d data_size 0x6c(0xb784 bytes) 2024-09-14T07:00:39.779 INFO:teuthology.orchestra.run.smithi007.stdout:4858 punch from 0x130b12c to 0x1314038, (0x8f0c bytes) 2024-09-14T07:00:39.779 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.779 INFO:teuthology.orchestra.run.smithi007.stdout:4860 read 0x627b762 thru 0x6284c5e (0x94fd bytes) 2024-09-14T07:00:39.779 INFO:teuthology.orchestra.run.smithi007.stdout:4861 read 0xcf06b9f thru 0xcf0b30e (0x4770 bytes) 2024-09-14T07:00:39.793 INFO:teuthology.orchestra.run.smithi007.stdout:4862 trunc from 0xd7bf032 to 0x1b7414d 2024-09-14T07:00:39.805 INFO:teuthology.orchestra.run.smithi007.stdout:4864 read 0xd75c57 thru 0xd7beed (0x6297 bytes) 2024-09-14T07:00:39.805 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.805 INFO:teuthology.orchestra.run.smithi007.stdout:4867 punch from 0xfeb250 to 0xff6d4d, (0xbafd bytes) 2024-09-14T07:00:39.805 INFO:teuthology.orchestra.run.smithi007.stdout:4868 punch from 0x17bd89b to 0x17c8667, (0xadcc bytes) 2024-09-14T07:00:39.806 INFO:teuthology.orchestra.run.smithi007.stdout:4869 read 0x13db78d thru 0x13e8085 (0xc8f9 bytes) 2024-09-14T07:00:39.811 INFO:teuthology.orchestra.run.smithi007.stdout:4870 write 0x1e2330e thru 0x1e300aa (0xcd9d bytes) 2024-09-14T07:00:39.815 INFO:teuthology.orchestra.run.smithi007.stdout:4871 read 0x11a89e1 thru 0x11b00b4 (0x76d4 bytes) 2024-09-14T07:00:39.831 INFO:teuthology.orchestra.run.smithi007.stdout:4872 write 0xb5c008d thru 0xb5c5d2d (0x5ca1 bytes) 2024-09-14T07:00:39.834 INFO:teuthology.orchestra.run.smithi007.stdout:4873 write 0x1dd4621 thru 0x1dd81ef (0x3bcf bytes) 2024-09-14T07:00:39.838 INFO:teuthology.orchestra.run.smithi007.stdout:4874 trunc from 0xb5c5d2e to 0xa114d99 2024-09-14T07:00:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.844 INFO:teuthology.orchestra.run.smithi007.stdout:4878 punch from 0x4849fd6 to 0x484eaa5, (0x4acf bytes) 2024-09-14T07:00:39.849 INFO:teuthology.orchestra.run.smithi007.stdout:4879 write 0xa210214 thru 0xa21d72a (0xd517 bytes) 2024-09-14T07:00:39.852 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:39.860 INFO:teuthology.orchestra.run.smithi007.stdout:4881 trunc from 0xa21d72b to 0x2f15a1d 2024-09-14T07:00:39.868 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:39.866+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:39.888 INFO:teuthology.orchestra.run.smithi007.stdout:4882 writesame 0xc7276a5 thru 0xc72e54f data_size 0x1(0x6eab bytes) 2024-09-14T07:00:39.888 INFO:teuthology.orchestra.run.smithi007.stdout:4883 read 0xa0d8795 thru 0xa0e4199 (0xba05 bytes) 2024-09-14T07:00:39.888 INFO:teuthology.orchestra.run.smithi007.stdout:4884 punch from 0x81a40e2 to 0x81b240c, (0xe32a bytes) 2024-09-14T07:00:39.888 INFO:teuthology.orchestra.run.smithi007.stdout:4885 punch from 0x461117 to 0x464325, (0x320e bytes) 2024-09-14T07:00:39.889 INFO:teuthology.orchestra.run.smithi007.stdout:4886 read 0x80e92d8 thru 0x80ec1f7 (0x2f20 bytes) 2024-09-14T07:00:39.889 INFO:teuthology.orchestra.run.smithi007.stdout:4887 punch from 0x72abd0a to 0x72b82e1, (0xc5d7 bytes) 2024-09-14T07:00:39.895 INFO:teuthology.orchestra.run.smithi007.stdout:4888 write 0xdedfea3 thru 0xdeecd47 (0xcea5 bytes) 2024-09-14T07:00:39.898 INFO:teuthology.orchestra.run.smithi007.stdout:4891 write 0xca5f34a thru 0xca64963 (0x561a bytes) 2024-09-14T07:00:39.901 INFO:teuthology.orchestra.run.smithi007.stdout:4892 write 0xf92afc thru 0xf9cbe1 (0xa0e6 bytes) 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.167412575, 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:00:39.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:00:39.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:00:39.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:00:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17618 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.664799637000002, 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016649290000000001, 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17618 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236332+0000", 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237113+0000", 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078082900000000003 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237131+0000", 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7747000000000001e-05 2024-09-14T07:00:39.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237141+0000", 2024-09-14T07:00:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0137e-05 2024-09-14T07:00:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237221+0000", 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0482000000000006e-05 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237383+0000", 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016176100000000001 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237737+0000", 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035380999999999998 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.241638+0000", 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039011739999999999 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252854+0000", 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011216097 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252919+0000", 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5129999999999995e-05 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252979+0000", 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9573e-05 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40284 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:9 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.696672575999997, 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016005640000000002, 2024-09-14T07:00:39.916 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40284 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204457+0000", 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204459+0000", 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-14T07:00:39.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204466+0000", 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8730000000000001e-06 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204472+0000", 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4779999999999999e-06 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207358+0000", 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028862549999999999 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207430+0000", 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2268000000000007e-05 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220462+0000", 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013031951999999999 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.920 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40646 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.695530941999998, 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017879991000000001, 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.921 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40646 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205605+0000", 2024-09-14T07:00:39.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2260000000000002e-06 2024-09-14T07:00:39.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205611+0000", 2024-09-14T07:00:39.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3289999999999999e-06 2024-09-14T07:00:39.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205622+0000", 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213726+0000", 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008104745 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213792+0000", 2024-09-14T07:00:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5401999999999995e-05 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223478+0000", 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096862100000000007 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40683 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.695459022999998, 2024-09-14T07:00:39.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016494009, 2024-09-14T07:00:39.927 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40683 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205672+0000", 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0379999999999998e-06 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205679+0000", 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205684+0000", 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9359999999999998e-06 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221845+0000", 2024-09-14T07:00:39.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016161422000000002 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221913+0000", 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7669000000000006e-05 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222164+0000", 2024-09-14T07:00:39.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025130600000000002 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40685 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.695438619000001, 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016657634000000001, 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40685 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:39.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205691+0000", 2024-09-14T07:00:39.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:00:39.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205693+0000", 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205699+0000", 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205703+0000", 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5240000000000002e-06 2024-09-14T07:00:39.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222190+0000", 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016486752 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222257+0000", 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7095999999999996e-05 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222348+0000", 2024-09-14T07:00:39.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1061000000000001e-05 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40687 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.695419389000001, 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016826543999999999, 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40687 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4999999999999998e-07 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205712+0000", 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205718+0000", 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:00:39.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205723+0000", 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222374+0000", 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016650933999999999 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222445+0000", 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1304999999999996e-05 2024-09-14T07:00:39.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222536+0000", 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1104000000000001e-05 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40720 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.695302793000003, 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016893371000000001, 2024-09-14T07:00:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40720 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205827+0000", 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.58e-07 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205834+0000", 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2729999999999997e-06 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205841+0000", 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205846+0000", 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1769999999999997e-06 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222562+0000", 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016716178000000002 2024-09-14T07:00:39.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222633+0000", 2024-09-14T07:00:39.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0461000000000002e-05 2024-09-14T07:00:39.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222720+0000", 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6997999999999996e-05 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40721 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.695275090999999, 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044822000000001, 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40721 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205857+0000", 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-06 2024-09-14T07:00:39.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205863+0000", 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205868+0000", 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222745+0000", 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016877293000000002 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222814+0000", 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8806000000000005e-05 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222899+0000", 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4913000000000005e-05 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40742 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.695254525000003, 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017201659000000001, 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40742 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205892+0000", 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7008e-05 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205899+0000", 2024-09-14T07:00:39.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1489999999999997e-06 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205904+0000", 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222923+0000", 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017018229999999999 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222991+0000", 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8545000000000002e-05 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223076+0000", 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4938999999999994e-05 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.956 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40743 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:39.956 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:39.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.695215793000003, 2024-09-14T07:00:39.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343654999999999, 2024-09-14T07:00:39.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40743 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205914+0000", 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-07 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205916+0000", 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0559999999999999e-06 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205922+0000", 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3659999999999997e-06 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205927+0000", 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8169999999999998e-06 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223101+0000", 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017174452 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223168+0000", 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6756999999999998e-05 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223257+0000", 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8700000000000001e-05 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43579 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.445267164000001, 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01522916, 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43579 2024-09-14T07:00:39.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455864+0000", 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455870+0000", 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455875+0000", 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9509999999999997e-06 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470879+0000", 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015003729 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470960+0000", 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1228000000000002e-05 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471091+0000", 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131018 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43580 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 110592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.445247625999997, 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015447047, 2024-09-14T07:00:39.965 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43580 2024-09-14T07:00:39.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.966 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455882+0000", 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T07:00:39.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455884+0000", 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2010000000000002e-06 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455891+0000", 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5690000000000001e-06 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455896+0000", 2024-09-14T07:00:39.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3639999999999998e-06 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471123+0000", 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015226805 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471200+0000", 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7405999999999997e-05 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471328+0000", 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012835099999999999 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43581 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.445225725, 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015668949000000001, 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:39.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43581 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455904+0000", 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455910+0000", 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6619999999999999e-06 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455917+0000", 2024-09-14T07:00:39.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5769999999999999e-06 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455923+0000", 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6779999999999997e-06 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471360+0000", 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015436854999999999 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471443+0000", 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3815999999999998e-05 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471572+0000", 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012892900000000001 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117541 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.009523789999999, 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015320307999999999, 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117541 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:39.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891606+0000", 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891609+0000", 2024-09-14T07:00:39.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.886e-06 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891615+0000", 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5010000000000001e-06 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891620+0000", 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901891+0000", 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010270833 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901942+0000", 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0612000000000003e-05 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902142+0000", 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020015500000000001 2024-09-14T07:00:39.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904269+0000", 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021265519999999999 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906874+0000", 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026050800000000001 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:39.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906904+0000", 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199e-05 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906926+0000", 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1708e-05 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117542 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:39.981 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.009502091000002, 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015409499, 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117542 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891630+0000", 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8569999999999999e-06 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891638+0000", 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5610000000000001e-06 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891643+0000", 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4299999999999997e-06 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902805+0000", 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011162071000000001 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902879+0000", 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4152000000000004e-05 2024-09-14T07:00:39.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903129+0000", 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024969499999999997 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904277+0000", 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001147906 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906968+0000", 2024-09-14T07:00:39.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002690942 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906999+0000", 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1056000000000002e-05 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.907036+0000", 2024-09-14T07:00:39.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7410000000000003e-05 2024-09-14T07:00:39.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117543 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:39.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:39.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.009469631000002, 2024-09-14T07:00:39.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018002384999999999, 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117543 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891660+0000", 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8100000000000003e-07 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891663+0000", 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2739999999999999e-06 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891671+0000", 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.303e-06 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891678+0000", 2024-09-14T07:00:39.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3709999999999996e-06 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903501+0000", 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011823382 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903577+0000", 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5947999999999999e-05 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903855+0000", 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027763899999999999 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904798+0000", 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094325599999999996 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909603+0000", 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048047539999999996 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909634+0000", 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1137999999999999e-05 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909662+0000", 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7838999999999999e-05 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:39.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:120396 3.1c 3:3ebb3b40:::rbd_object_map.114f74d20483.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.342504914999999, 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015664009999999999, 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 120396 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.995 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558625+0000", 2024-09-14T07:00:39.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-14T07:00:39.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:39.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558631+0000", 2024-09-14T07:00:39.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:39.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558647+0000", 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5756999999999998e-05 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558657+0000", 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.995e-06 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558724+0000", 2024-09-14T07:00:39.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7008000000000006e-05 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558815+0000", 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1077999999999998e-05 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.559173+0000", 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035768600000000002 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:39.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560127+0000", 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095468799999999998 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574168+0000", 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014040238999999999 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574233+0000", 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999994e-05 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574288+0000", 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.558e-05 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:134959 3.1a 3:59279a6c:::rbd_data.114fdc2d0201.0000000000000072:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:21.970619+0000", 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.930510261999999, 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021043874000000001, 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 134959 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970619+0000", 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970619+0000", 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970620+0000", 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970625+0000", 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5450000000000003e-06 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970638+0000", 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2777e-05 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970646+0000", 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9680000000000006e-06 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970714+0000", 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8361999999999994e-05 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970796+0000", 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1577999999999997e-05 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.974492+0000", 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036963090000000001 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991537+0000", 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044678000000001 2024-09-14T07:00:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991586+0000", 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8689000000000001e-05 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991627+0000", 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1085999999999999e-05 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991663+0000", 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5936999999999998e-05 2024-09-14T07:00:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138654 3.1c 3:3ed33fa1:::rbd_data.114f74bd3bbf.00000000000000cc:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:31.531314+0000", 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.3698148289999992, 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016015504, 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138654 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531314+0000", 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531314+0000", 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531315+0000", 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000004e-07 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531319+0000", 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1999999999999996e-06 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531330+0000", 2024-09-14T07:00:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0414999999999999e-05 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531336+0000", 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2389999999999999e-06 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531382+0000", 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5867e-05 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531474+0000", 2024-09-14T07:00:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2489000000000002e-05 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.534588+0000", 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031140849999999999 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547207+0000", 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012618994999999999 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547261+0000", 2024-09-14T07:00:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3269000000000001e-05 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547298+0000", 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7471000000000001e-05 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547330+0000", 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1680000000000002e-05 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.013 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:40.013 INFO:teuthology.orchestra.run.smithi007.stdout:4894 writesame 0x37f192b thru 0x37f3fd9 data_size 0x1(0x26af bytes) 2024-09-14T07:00:40.013 INFO:teuthology.orchestra.run.smithi007.stdout:4895 trunc from 0xdeecd48 to 0xccf3358 2024-09-14T07:00:40.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:40.016 INFO:teuthology.orchestra.run.smithi007.stdout:4896 writesame 0x875aa96 thru 0x87602ab data_size 0x37(0x5816 bytes) 2024-09-14T07:00:40.016 INFO:teuthology.orchestra.run.smithi007.stdout:4897 read 0x4499624 thru 0x44a452e (0xaf0b bytes) 2024-09-14T07:00:40.016 INFO:teuthology.orchestra.run.smithi007.stdout:4898 punch from 0x233e4fa to 0x234414c, (0x5c52 bytes) 2024-09-14T07:00:40.016 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:40.016 INFO:teuthology.orchestra.run.smithi007.stdout:4902 read 0x1e64e1c thru 0x1e71f7c (0xd161 bytes) 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4903 read 0xc7b72f5 thru 0xc7b8164 (0xe70 bytes) 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4905 read 0x156e509 thru 0x1574aab (0x65a3 bytes) 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4908 punch from 0x286a4e5 to 0x2879363, (0xee7e bytes) 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4910 writesame 0x9b94607 thru 0x9b9f430 data_size 0x9(0xae2a bytes) 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4911 read 0x4521b27 thru 0x4523a80 (0x1f5a bytes) 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4913 punch from 0x9379413 to 0x9383c76, (0xa863 bytes) 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4914 trunc from 0xccf3358 to 0x3f0cf91 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4915 read 0x1c6b87a thru 0x1c6ecc9 (0x3450 bytes) 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4916 read 0x2040663 thru 0x204d2a7 (0xcc45 bytes) 2024-09-14T07:00:40.017 INFO:teuthology.orchestra.run.smithi007.stdout:4919 clone 30 order 18 su 2048 sc 6 2024-09-14T07:00:40.056 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:40.055+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:40.117 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:40.117+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:40.163 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:40.164 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:40.164 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:40.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:40.219 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:40.218+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:40.281 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:40.281 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:40.281 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:40.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:40.319 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:40.319+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:40.395 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119746 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1298432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.849343305000001, 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067798875999999994, 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119746 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545019+0000", 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545020+0000", 2024-09-14T07:00:40.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.782e-06 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545026+0000", 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9390000000000002e-06 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545031+0000", 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5770000000000004e-06 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612601+0000", 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067570682000000007 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612683+0000", 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1378999999999995e-05 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612817+0000", 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013415000000000001 2024-09-14T07:00:40.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119747 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.849324886000002, 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068036012000000007, 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.401 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119747 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:40.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546792+0000", 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017554960000000001 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546797+0000", 2024-09-14T07:00:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1950000000000002e-06 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546808+0000", 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0675e-05 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546815+0000", 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612854+0000", 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066039158000000001 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612930+0000", 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.613073+0000", 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000142798 2024-09-14T07:00:40.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119753 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1398784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.847400724, 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067577770999999995, 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119753 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546993+0000", 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1860999999999999e-05 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547007+0000", 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4346e-05 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547016+0000", 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4709999999999996e-06 2024-09-14T07:00:40.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547023+0000", 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4010000000000002e-06 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614322+0000", 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067299231000000001 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614391+0000", 2024-09-14T07:00:40.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8826999999999997e-05 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614538+0000", 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000147634 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119754 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1413120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.847310186000001, 2024-09-14T07:00:40.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067735654000000006, 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119754 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549973+0000", 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029215999999999999 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549978+0000", 2024-09-14T07:00:40.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3510000000000001e-06 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549988+0000", 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0037000000000001e-05 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549995+0000", 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9709999999999998e-06 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614569+0000", 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064573368000000006 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614650+0000", 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1909000000000005e-05 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614787+0000", 2024-09-14T07:00:40.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013641799999999999 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119764 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1556480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.844138278999999, 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067672107999999995, 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119764 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550224+0000", 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550226+0000", 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1720000000000001e-06 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550233+0000", 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9879999999999998e-06 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550238+0000", 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1819999999999997e-06 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617715+0000", 2024-09-14T07:00:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067476628999999996 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617807+0000", 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2899999999999995e-05 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617895+0000", 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7837999999999995e-05 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119765 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.844115847000001, 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067829500000000001, 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119765 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7e-07 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550248+0000", 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1280000000000002e-06 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550256+0000", 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550261+0000", 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617926+0000", 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067665280999999994 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617995+0000", 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9478000000000001e-05 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618075+0000", 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9666000000000006e-05 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119950 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 529408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.844093741000002, 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067992002999999995, 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119950 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550852+0000", 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058435400000000004 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550857+0000", 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6979999999999997e-06 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550867+0000", 2024-09-14T07:00:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0378e-05 2024-09-14T07:00:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550874+0000", 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8480000000000003e-06 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618106+0000", 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067231714999999997 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618178+0000", 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2572999999999998e-05 2024-09-14T07:00:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618260+0000", 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1437e-05 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119970 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 816128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837438585000001, 2024-09-14T07:00:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067745828999999994, 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119970 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556924+0000", 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0410000000000001e-06 2024-09-14T07:00:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556927+0000", 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9679999999999998e-06 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556930+0000", 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624487+0000", 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067557392999999993 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624551+0000", 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3861000000000004e-05 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624669+0000", 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117921 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119971 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837428645999999, 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067869065000000006, 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119971 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556934+0000", 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.096e-06 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556937+0000", 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.796e-06 2024-09-14T07:00:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556939+0000", 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624708+0000", 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067768992 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624747+0000", 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8148999999999997e-05 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624802+0000", 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.533e-05 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119972 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 844800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837418974999999, 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068010762000000002, 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119972 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556943+0000", 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556944+0000", 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556947+0000", 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9720000000000001e-06 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556949+0000", 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4899999999999999e-06 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624838+0000", 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067888899000000003 2024-09-14T07:00:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624895+0000", 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7395000000000002e-05 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624953+0000", 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7745999999999999e-05 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119973 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 859136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837409093000002, 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068210898000000006, 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119973 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556953+0000", 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556954+0000", 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189999999999999e-06 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556957+0000", 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9490000000000001e-06 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556959+0000", 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3870000000000002e-06 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624983+0000", 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068023726000000007 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625049+0000", 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6385999999999998e-05 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625163+0000", 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114221 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119974 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837399172000001, 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068338594000000003, 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119974 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556963+0000", 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556964+0000", 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556969+0000", 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.04e-06 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556974+0000", 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.639e-06 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625194+0000", 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068220742000000001 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625246+0000", 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1844e-05 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625301+0000", 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4551e-05 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119975 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837380931999999, 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068554198999999996, 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119975 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556983+0000", 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2560000000000001e-06 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556989+0000", 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2210000000000002e-06 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556994+0000", 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7779999999999998e-06 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625330+0000", 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068335911999999999 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625396+0000", 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6051999999999997e-05 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625535+0000", 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013860200000000001 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119976 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837359936999999, 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068656164000000006, 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119976 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557004+0000", 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557010+0000", 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1770000000000003e-06 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557016+0000", 2024-09-14T07:00:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2650000000000004e-06 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625566+0000", 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068550323999999996 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625604+0000", 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8398000000000001e-05 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625658+0000", 2024-09-14T07:00:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3239000000000003e-05 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119977 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837279517999999, 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068713522999999999, 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119977 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557086+0000", 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3569999999999998e-06 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557094+0000", 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6829999999999997e-06 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557101+0000", 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625689+0000", 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068587882000000003 2024-09-14T07:00:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625743+0000", 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4082000000000002e-05 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625795+0000", 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2657000000000001e-05 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119978 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 930816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837250925999999, 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068826410000000005, 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119978 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:00:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557113+0000", 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557120+0000", 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1079999999999998e-06 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557125+0000", 2024-09-14T07:00:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625823+0000", 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068698928000000006 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625884+0000", 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0056999999999998e-05 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625937+0000", 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3383e-05 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119979 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 945152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837229779000001, 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068940788000000003, 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119979 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5199999999999998e-07 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557134+0000", 2024-09-14T07:00:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.018e-06 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557140+0000", 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557145+0000", 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7790000000000003e-06 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625966+0000", 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068820972999999994 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626019+0000", 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3828e-05 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626072+0000", 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3078000000000002e-05 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119980 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837209421000001, 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069053630000000005, 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119980 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557159+0000", 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557167+0000", 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999996e-06 2024-09-14T07:00:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557172+0000", 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626101+0000", 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068928552000000004 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626154+0000", 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3338000000000003e-05 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626206+0000", 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1657999999999999e-05 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119981 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837180771, 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069157629999999998, 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119981 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557183+0000", 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557190+0000", 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2249999999999997e-06 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557195+0000", 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3120000000000001e-06 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626234+0000", 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069038926 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626286+0000", 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2604e-05 2024-09-14T07:00:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626338+0000", 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1947000000000003e-05 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119982 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 988160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.837159618000001, 2024-09-14T07:00:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069364004000000007, 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119982 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560989+0000", 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037867360000000002 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560994+0000", 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561004+0000", 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3680000000000008e-06 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561011+0000", 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1640000000000004e-06 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626366+0000", 2024-09-14T07:00:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.065355286999999998 2024-09-14T07:00:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626433+0000", 2024-09-14T07:00:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6554e-05 2024-09-14T07:00:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626566+0000", 2024-09-14T07:00:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132996 2024-09-14T07:00:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.489 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:40.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:40.491 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:40.490+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:40.591 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:40.589+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:142508 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=60b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:40.604454+0000", 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001352991, 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013798910000000001, 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:40.606 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 142508 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.604454+0000", 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.604454+0000", 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.604456+0000", 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0959999999999999e-06 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.604465+0000", 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6009999999999995e-06 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.604486+0000", 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1291999999999999e-05 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.604497+0000", 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.057e-05 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.604558+0000", 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0751000000000002e-05 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.604616+0000", 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8400000000000003e-05 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.604754+0000", 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013801199999999999 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605608+0000", 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00085354300000000001 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605738+0000", 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000130798 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605793+0000", 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4258999999999997e-05 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605846+0000", 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3665000000000002e-05 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop_reply(client.4431.0:142508 3.a e87/24 ondisk, result = 0)", 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:40.605660+0000", 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.00014688900000000001, 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023463999999999999, 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605660+0000", 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605660+0000", 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605661+0000", 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6899999999999996e-07 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605665+0000", 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7840000000000001e-06 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605674+0000", 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3570000000000004e-06 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605679+0000", 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9570000000000001e-06 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605722+0000", 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2744000000000003e-05 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:40.605731+0000", 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1609999999999992e-06 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.617 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:40.617 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 2 2024-09-14T07:00:40.617 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:40.618 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:40.684 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-14T07:00:40.692 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:40.691+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:40.695 INFO:teuthology.orchestra.run.smithi007.stdout:4920 write 0x26cdeee thru 0x26d58f9 (0x7a0c bytes) 2024-09-14T07:00:40.782 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:40.782 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:40.783 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:40.783 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:40.783 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:40.791 INFO:teuthology.orchestra.run.smithi007.stdout:4922 write 0x430af77 thru 0x4315607 (0xa691 bytes) 2024-09-14T07:00:40.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:40.793 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:40.792+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:40.815 INFO:teuthology.orchestra.run.smithi007.stdout:4923 write 0xcda103e thru 0xcdab654 (0xa617 bytes) 2024-09-14T07:00:40.823 INFO:teuthology.orchestra.run.smithi007.stdout:4925 write 0x42cda84 thru 0x42d8040 (0xa5bd bytes) 2024-09-14T07:00:40.825 INFO:teuthology.orchestra.run.smithi007.stdout:4927 read 0x383db7b thru 0x3847445 (0x98cb bytes) 2024-09-14T07:00:40.828 INFO:teuthology.orchestra.run.smithi007.stdout:4928 punch from 0x6e898db to 0x6e93990, (0xa0b5 bytes) 2024-09-14T07:00:40.835 INFO:teuthology.orchestra.run.smithi007.stdout:4930 writesame 0xd7f39cc thru 0xd8039bd data_size 0x2(0xfff2 bytes) 2024-09-14T07:00:40.835 INFO:teuthology.orchestra.run.smithi007.stdout:4931 read 0x4666f14 thru 0x4676812 (0xf8ff bytes) 2024-09-14T07:00:40.835 INFO:teuthology.orchestra.run.smithi007.stdout:4932 punch from 0x355db90 to 0x356497c, (0x6dec bytes) 2024-09-14T07:00:40.835 INFO:teuthology.orchestra.run.smithi007.stdout:4933 writesame 0x8b05713 thru 0x8b10e02 data_size 0x8(0xb6f0 bytes) 2024-09-14T07:00:40.844 INFO:teuthology.orchestra.run.smithi007.stdout:4935 read 0xc652d27 thru 0xc65a1e1 (0x74bb bytes) 2024-09-14T07:00:40.844 INFO:teuthology.orchestra.run.smithi007.stdout:4937 punch from 0xa1f20c2 to 0xa1ff77e, (0xd6bc bytes) 2024-09-14T07:00:40.849 INFO:teuthology.orchestra.run.smithi007.stdout:4938 write 0xd9d6901 thru 0xd9db883 (0x4f83 bytes) 2024-09-14T07:00:40.858 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:40.858 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:40.858 INFO:teuthology.orchestra.run.smithi007.stdout:4941 read 0x9971c28 thru 0x997d132 (0xb50b bytes) 2024-09-14T07:00:40.858 INFO:teuthology.orchestra.run.smithi007.stdout:4942 writesame 0x8ce8059 thru 0x8cf1f92 data_size 0x59(0x9f3a bytes) 2024-09-14T07:00:40.866 INFO:teuthology.orchestra.run.smithi007.stdout:4943 read 0x38ecf35 thru 0x38eeb0d (0x1bd9 bytes) 2024-09-14T07:00:40.867 INFO:teuthology.orchestra.run.smithi007.stdout:4944 read 0x33bc85d thru 0x33bd3c5 (0xb69 bytes) 2024-09-14T07:00:40.867 INFO:teuthology.orchestra.run.smithi007.stdout:4945 write 0x46b6e8b thru 0x46c105b (0xa1d1 bytes) 2024-09-14T07:00:40.875 INFO:teuthology.orchestra.run.smithi007.stdout:4946 read 0x261375a thru 0x261f999 (0xc240 bytes) 2024-09-14T07:00:40.876 INFO:teuthology.orchestra.run.smithi007.stdout:4947 read 0x5374f91 thru 0x53815e1 (0xc651 bytes) 2024-09-14T07:00:40.878 INFO:teuthology.orchestra.run.smithi007.stdout:4950 trunc from 0xd9db884 to 0xc32cf2d 2024-09-14T07:00:40.889 INFO:teuthology.orchestra.run.smithi007.stdout:4951 punch from 0x6ecb525 to 0x6ed08d9, (0x53b4 bytes) 2024-09-14T07:00:40.896 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:40.893+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:40.897 INFO:teuthology.orchestra.run.smithi007.stdout:4952 trunc from 0xc32cf2d to 0x579f116 2024-09-14T07:00:40.914 INFO:teuthology.orchestra.run.smithi007.stdout:4953 read 0x47b73a7 thru 0x47be16e (0x6dc8 bytes) 2024-09-14T07:00:40.914 INFO:teuthology.orchestra.run.smithi007.stdout:4954 write 0x3d1e995 thru 0x3d2ced6 (0xe542 bytes) 2024-09-14T07:00:40.949 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.194328265999999, 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:00:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.595915673999997, 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:00:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.590256031000003, 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:00:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:00:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:00:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.284419753000002, 2024-09-14T07:00:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:00:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.284388960999998, 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:00:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.284365590999997, 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.283559081999996, 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.283536689000002, 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.283516548999998, 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:00:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:00:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.283366393999998, 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.283310839999999, 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:00:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.283277798999997, 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:00:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:00:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.280924714000001, 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.280902148999999, 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:00:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.280045154, 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:00:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.278828132999998, 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:00:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.278700293, 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.278680199999997, 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:00:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.277653307000001, 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138497 3.14 3:298b17d3:::rbd_data.114f74bd3bbf.00000000000002d7:head [set-alloc-hint object_size 262144 write_size 262144,write 218355~43789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-14T07:00:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.630146025, 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97686345699999999, 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138497 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317983+0000", 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:00:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318003+0000", 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0610000000000001e-05 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318022+0000", 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8627e-05 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318030+0000", 2024-09-14T07:00:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318101+0000", 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1162999999999996e-05 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318108+0000", 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291257+0000", 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97314914100000005 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291396+0000", 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013883799999999999 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291573+0000", 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017716800000000001 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.293147+0000", 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015740400000000001 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294753+0000", 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00160597 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294792+0000", 2024-09-14T07:00:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8758999999999999e-05 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294844+0000", 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2188000000000002e-05 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:41.048 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:41.050 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stdout:4955 punch from 0x2885f96 to 0x288eecf, (0x8f39 bytes) 2024-09-14T07:00:41.050 INFO:teuthology.orchestra.run.smithi007.stdout:4956 trunc from 0x579f116 to 0x2eae582 2024-09-14T07:00:41.051 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:41.048+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:41.091 INFO:teuthology.orchestra.run.smithi007.stdout:4958 read 0x1e421e6 thru 0x1e42dad (0xbc8 bytes) 2024-09-14T07:00:41.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:41.102 INFO:teuthology.orchestra.run.smithi007.stdout:4960 writesame 0x4a710f8 thru 0x4a74cd6 data_size 0x27(0x3bdf bytes) 2024-09-14T07:00:41.108 INFO:teuthology.orchestra.run.smithi007.stdout:4961 write 0x7aae2d7 thru 0x7abb802 (0xd52c bytes) 2024-09-14T07:00:41.117 INFO:teuthology.orchestra.run.smithi007.stdout:4962 punch from 0x12d4148 to 0x12db9b5, (0x786d bytes) 2024-09-14T07:00:41.124 INFO:teuthology.orchestra.run.smithi007.stdout:4963 write 0x9d53492 thru 0x9d54b29 (0x1698 bytes) 2024-09-14T07:00:41.130 INFO:teuthology.orchestra.run.smithi007.stdout:4964 punch from 0x21848d to 0x223166, (0xacd9 bytes) 2024-09-14T07:00:41.130 INFO:teuthology.orchestra.run.smithi007.stdout:4965 write 0x1cc9e4c thru 0x1ccfec0 (0x6075 bytes) 2024-09-14T07:00:41.150 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:41.150+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:41.173 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:41.173 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:41.173 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:41.173 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:41.184 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:41.214 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.214 INFO:teuthology.orchestra.run.smithi007.stdout:4968 write 0x99dcfe5 thru 0x99e952b (0xc547 bytes) 2024-09-14T07:00:41.222 INFO:teuthology.orchestra.run.smithi007.stdout:4969 read 0x1e9abc9 thru 0x1eaa173 (0xf5ab bytes) 2024-09-14T07:00:41.228 INFO:teuthology.orchestra.run.smithi007.stdout:4970 writesame 0x2353ab2 thru 0x235ed0f data_size 0x4f(0xb25e bytes) 2024-09-14T07:00:41.251 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:41.249+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:41.274 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:41.275 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:41.309 INFO:teuthology.orchestra.run.smithi007.stdout:4971 read 0x9d0ca26 thru 0x9d14d84 (0x835f bytes) 2024-09-14T07:00:41.310 INFO:teuthology.orchestra.run.smithi007.stdout:4973 write 0x7c50556 thru 0x7c5f5d9 (0xf084 bytes) 2024-09-14T07:00:41.318 INFO:teuthology.orchestra.run.smithi007.stdout:4974 read 0x4fef8e2 thru 0x4ff7639 (0x7d58 bytes) 2024-09-14T07:00:41.318 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.318 INFO:teuthology.orchestra.run.smithi007.stdout:4978 write 0x4f40f84 thru 0x4f4f6d8 (0xe755 bytes) 2024-09-14T07:00:41.330 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:41.330 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:41.330 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:41.330 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:41.330 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:41.331 INFO:teuthology.orchestra.run.smithi007.stdout:4979 trunc from 0x9d54b2a to 0x4cfa13b 2024-09-14T07:00:41.340 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:41.347 INFO:teuthology.orchestra.run.smithi007.stdout:4980 read 0x17352d6 thru 0x17365a9 (0x12d4 bytes) 2024-09-14T07:00:41.360 INFO:teuthology.orchestra.run.smithi007.stdout:4981 write 0xb4cafca thru 0xb4ce8d2 (0x3909 bytes) 2024-09-14T07:00:41.368 INFO:teuthology.orchestra.run.smithi007.stdout:4982 read 0xada76e7 thru 0xadb100c (0x9926 bytes) 2024-09-14T07:00:41.377 INFO:teuthology.orchestra.run.smithi007.stdout:4983 trunc from 0xb4ce8d3 to 0x2e3eafe 2024-09-14T07:00:41.384 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:41.384+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:41.453 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:41.451+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:41.477 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:41.477 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:41.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:41.477 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:41.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.477 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131358 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.477 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:41.477 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977415162, 2024-09-14T07:00:41.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030805583, 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131358 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498000+0000", 2024-09-14T07:00:41.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498004+0000", 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2569999999999999e-06 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498007+0000", 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.507e-06 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499958+0000", 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001950989 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499962+0000", 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528792+0000", 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288306389999999 2024-09-14T07:00:41.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528804+0000", 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2096000000000001e-05 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131359 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977395752, 2024-09-14T07:00:41.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308363920000001, 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131359 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498019+0000", 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498020+0000", 2024-09-14T07:00:41.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4529999999999999e-06 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498023+0000", 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3450000000000002e-06 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498026+0000", 2024-09-14T07:00:41.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.661e-06 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499995+0000", 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00196926 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500002+0000", 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6930000000000001e-06 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528842+0000", 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288401300000001 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528855+0000", 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2595999999999999e-05 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131360 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:41.487 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977384748999999, 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030841943, 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131360 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:41.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498030+0000", 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498031+0000", 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1119999999999999e-06 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498034+0000", 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498036+0000", 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.599e-06 2024-09-14T07:00:41.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500030+0000", 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001993613 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500034+0000", 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1060000000000002e-06 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528867+0000", 2024-09-14T07:00:41.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288327429999999 2024-09-14T07:00:41.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528871+0000", 2024-09-14T07:00:41.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.673e-06 2024-09-14T07:00:41.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131361 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977364004000002, 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308644520000001, 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131361 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498052+0000", 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.24e-06 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498055+0000", 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1609999999999999e-06 2024-09-14T07:00:41.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498058+0000", 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.05e-06 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500066+0000", 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020081169999999998 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500075+0000", 2024-09-14T07:00:41.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0559999999999994e-06 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528903+0000", 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288276810000001 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528915+0000", 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1938e-05 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131362 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977345223, 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030860661, 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131362 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.498 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:41.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498075+0000", 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5639999999999996e-06 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498079+0000", 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498081+0000", 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8660000000000002e-06 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500101+0000", 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020191219999999999 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500104+0000", 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6890000000000002e-06 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528925+0000", 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028820971 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528930+0000", 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3390000000000003e-06 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.502 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131363 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977328974999999, 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030874107, 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.503 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131363 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498086+0000", 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1009999999999999e-06 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498090+0000", 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5609999999999999e-06 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498093+0000", 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7360000000000001e-06 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500136+0000", 2024-09-14T07:00:41.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020436030000000002 2024-09-14T07:00:41.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500145+0000", 2024-09-14T07:00:41.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9530000000000005e-06 2024-09-14T07:00:41.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528948+0000", 2024-09-14T07:00:41.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028803103 2024-09-14T07:00:41.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528959+0000", 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0808999999999999e-05 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131364 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:41.509 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977312594000001, 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308734129999999, 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131364 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:41.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:41.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498102+0000", 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498103+0000", 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498107+0000", 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4089999999999999e-06 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498110+0000", 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7760000000000002e-06 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500177+0000", 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020678200000000002 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500183+0000", 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4979999999999997e-06 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528970+0000", 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287867049999999 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528975+0000", 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1959999999999999e-06 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131365 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977290103000001, 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886024, 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.517 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131365 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498125+0000", 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.17e-06 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498129+0000", 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:00:41.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498131+0000", 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.666e-06 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500210+0000", 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020789559999999999 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500215+0000", 2024-09-14T07:00:41.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528993+0000", 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028777174 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529010+0000", 2024-09-14T07:00:41.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7419999999999999e-05 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131366 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977271805000001, 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308831409999999, 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.522 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131366 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498143+0000", 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498144+0000", 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.474e-06 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498147+0000", 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.371e-06 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498150+0000", 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6410000000000002e-06 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500239+0000", 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002088842 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500243+0000", 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6689999999999999e-06 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529020+0000", 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287776420000001 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529025+0000", 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2440000000000001e-06 2024-09-14T07:00:41.526 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131367 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977260744999999, 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030895135, 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131367 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498154+0000", 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2100000000000001e-07 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498155+0000", 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1310000000000001e-06 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498157+0000", 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7870000000000002e-06 2024-09-14T07:00:41.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498160+0000", 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9009999999999998e-06 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500264+0000", 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021031489999999999 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500268+0000", 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0929999999999996e-06 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529043+0000", 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287752139999999 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529048+0000", 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6389999999999997e-06 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131368 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977236612999999, 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308865389999999, 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.532 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131368 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498178+0000", 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8200000000000001e-07 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498179+0000", 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6929999999999999e-06 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498183+0000", 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9439999999999998e-06 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498188+0000", 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9409999999999998e-06 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500287+0000", 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020982269999999998 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500290+0000", 2024-09-14T07:00:41.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6349999999999999e-06 2024-09-14T07:00:41.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529059+0000", 2024-09-14T07:00:41.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028768648 2024-09-14T07:00:41.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529064+0000", 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.169e-06 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131369 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977221420999999, 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886017, 2024-09-14T07:00:41.537 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131369 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.28e-07 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498194+0000", 2024-09-14T07:00:41.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1459999999999999e-06 2024-09-14T07:00:41.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498197+0000", 2024-09-14T07:00:41.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0620000000000001e-06 2024-09-14T07:00:41.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498200+0000", 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6000000000000001e-06 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500310+0000", 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021101840000000002 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500314+0000", 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.126e-06 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529074+0000", 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.02875961 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529079+0000", 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0610000000000003e-06 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.542 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131370 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977205628, 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308930119999999, 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131370 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498209+0000", 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498210+0000", 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.187e-06 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498213+0000", 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.151e-06 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498216+0000", 2024-09-14T07:00:41.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8289999999999999e-06 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500340+0000", 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021244229999999998 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500349+0000", 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529096+0000", 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287473309999999 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529101+0000", 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131371 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:41.547 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977187248, 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308901880000001, 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131371 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:00:41.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498229+0000", 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1110000000000002e-06 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498234+0000", 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498239+0000", 2024-09-14T07:00:41.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.651e-06 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500382+0000", 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021424780000000002 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500389+0000", 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529112+0000", 2024-09-14T07:00:41.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287229410000001 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529117+0000", 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9520000000000002e-06 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131372 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:41.552 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977166132000001, 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030884366, 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131372 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:41.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:41.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8599999999999999e-07 2024-09-14T07:00:41.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498251+0000", 2024-09-14T07:00:41.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:00:41.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498257+0000", 2024-09-14T07:00:41.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6370000000000002e-06 2024-09-14T07:00:41.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498262+0000", 2024-09-14T07:00:41.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:41.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500418+0000", 2024-09-14T07:00:41.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021561800000000002 2024-09-14T07:00:41.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500423+0000", 2024-09-14T07:00:41.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1520000000000002e-06 2024-09-14T07:00:41.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529127+0000", 2024-09-14T07:00:41.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287049150000001 2024-09-14T07:00:41.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529132+0000", 2024-09-14T07:00:41.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.865e-06 2024-09-14T07:00:41.559 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.559 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.559 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.559 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131373 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.559 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:41.560 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977133069000001, 2024-09-14T07:00:41.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308673479999999, 2024-09-14T07:00:41.560 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.560 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.560 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.560 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.560 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.560 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.561 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131373 2024-09-14T07:00:41.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.561 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:41.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:41.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:41.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:00:41.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498283+0000", 2024-09-14T07:00:41.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:41.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498290+0000", 2024-09-14T07:00:41.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:00:41.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498295+0000", 2024-09-14T07:00:41.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3759999999999999e-06 2024-09-14T07:00:41.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500441+0000", 2024-09-14T07:00:41.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021462849999999999 2024-09-14T07:00:41.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500445+0000", 2024-09-14T07:00:41.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8310000000000003e-06 2024-09-14T07:00:41.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529142+0000", 2024-09-14T07:00:41.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028697363 2024-09-14T07:00:41.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529148+0000", 2024-09-14T07:00:41.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:41.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.567 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.567 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131374 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.567 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977099453000001, 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308477090000001, 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131374 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:00:41.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498322+0000", 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5760000000000002e-06 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498328+0000", 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4609999999999996e-06 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498334+0000", 2024-09-14T07:00:41.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7130000000000002e-06 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500464+0000", 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002130617 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500468+0000", 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7340000000000002e-06 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529157+0000", 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286893669999999 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529162+0000", 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131375 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.572 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977059315999998, 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308296320000001, 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131375 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:41.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498358+0000", 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.317e-06 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498364+0000", 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5930000000000002e-06 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498369+0000", 2024-09-14T07:00:41.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3630000000000002e-06 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500488+0000", 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021187699999999999 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500492+0000", 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529179+0000", 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286873590000001 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529184+0000", 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131376 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.977031777000001, 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030817713, 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.578 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131376 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498383+0000", 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498385+0000", 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.091e-06 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498391+0000", 2024-09-14T07:00:41.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1070000000000001e-06 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498396+0000", 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4670000000000003e-06 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500514+0000", 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021177449999999999 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500518+0000", 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8770000000000003e-06 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529195+0000", 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028676938 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529200+0000", 2024-09-14T07:00:41.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131377 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.976992251999999, 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308008749999999, 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:41.583 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131377 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:41.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498425+0000", 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5620000000000002e-06 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498431+0000", 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2539999999999998e-06 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498436+0000", 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.31e-06 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500539+0000", 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021025459999999998 2024-09-14T07:00:41.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500543+0000", 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.597e-06 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529218+0000", 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286753930000001 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529223+0000", 2024-09-14T07:00:41.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.758e-06 2024-09-14T07:00:41.588 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.588 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.588 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.588 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:41.588 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:41.588 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:41.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:00:41.590 INFO:teuthology.orchestra.run.smithi007.stdout:4984 write 0x5935177 thru 0x5936b8c (0x1a16 bytes) 2024-09-14T07:00:41.590 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.590 INFO:teuthology.orchestra.run.smithi007.stdout:4986 punch from 0x5490e2f to 0x54a0924, (0xfaf5 bytes) 2024-09-14T07:00:41.590 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.590 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.590 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:41.591 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:41.591 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:41.592 INFO:teuthology.orchestra.run.smithi007.stdout:4991 write 0xd64b395 thru 0xd64f6a6 (0x4312 bytes) 2024-09-14T07:00:41.592 INFO:teuthology.orchestra.run.smithi007.stdout:4992 write 0x4473eae thru 0x447c1a1 (0x82f4 bytes) 2024-09-14T07:00:41.592 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:41.593 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:41.594 INFO:teuthology.orchestra.run.smithi007.stdout:4993 punch from 0x3f03c2e to 0x3f0e53f, (0xa911 bytes) 2024-09-14T07:00:41.594 INFO:teuthology.orchestra.run.smithi007.stdout:4994 punch from 0x591dde7 to 0x592423c, (0x6455 bytes) 2024-09-14T07:00:41.594 INFO:teuthology.orchestra.run.smithi007.stdout:4995 punch from 0x17bb26a to 0x17c3759, (0x84ef bytes) 2024-09-14T07:00:41.594 INFO:teuthology.orchestra.run.smithi007.stdout:4996 write 0x3212d3c thru 0x321b6bd (0x8982 bytes) 2024-09-14T07:00:41.594 INFO:teuthology.orchestra.run.smithi007.stdout:4997 read 0xc464feb thru 0xc473f33 (0xef49 bytes) 2024-09-14T07:00:41.594 INFO:teuthology.orchestra.run.smithi007.stdout:5000 writesame 0x9be50d3 thru 0x9bea078 data_size 0x5(0x4fa6 bytes) 2024-09-14T07:00:41.594 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.594 INFO:teuthology.orchestra.run.smithi007.stdout:5002 write 0x985a9f6 thru 0x985ca43 (0x204e bytes) 2024-09-14T07:00:41.594 INFO:teuthology.orchestra.run.smithi007.stdout:5003 trunc from 0xd64f6a7 to 0xbea9d4c 2024-09-14T07:00:41.595 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:41.595 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:41.595 INFO:teuthology.orchestra.run.smithi007.stdout:5004 punch from 0x9c27800 to 0x9c32217, (0xaa17 bytes) 2024-09-14T07:00:41.595 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.596 INFO:teuthology.orchestra.run.smithi007.stdout:5006 write 0xa2427e3 thru 0xa24b9e0 (0x91fe bytes) 2024-09-14T07:00:41.596 INFO:teuthology.orchestra.run.smithi007.stdout:5007 trunc from 0xbea9d4c to 0xb6e0b45 2024-09-14T07:00:41.596 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:41.596 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:41.597 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.597 INFO:teuthology.orchestra.run.smithi007.stdout:5009 write 0xdf854e thru 0xdff4de (0x6f91 bytes) 2024-09-14T07:00:41.597 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:41.597 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:41.598 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:41.587+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:41.608 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:41.629 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:41.629 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:41.662 INFO:teuthology.orchestra.run.smithi007.stdout:5011 writesame 0x7ddb269 thru 0x7de199c data_size 0xa(0x6734 bytes) 2024-09-14T07:00:41.671 INFO:teuthology.orchestra.run.smithi007.stdout:5012 read 0x91e7e42 thru 0x91f70f4 (0xf2b3 bytes) 2024-09-14T07:00:41.672 INFO:teuthology.orchestra.run.smithi007.stdout:5013 writesame 0x8a2a011 thru 0x8a2f313 data_size 0x1(0x5303 bytes) 2024-09-14T07:00:41.682 INFO:teuthology.orchestra.run.smithi007.stdout:5015 read 0x1cf824e thru 0x1d0012d (0x7ee0 bytes) 2024-09-14T07:00:41.683 INFO:teuthology.orchestra.run.smithi007.stdout:5017 writesame 0xeddbff thru 0xee5992 data_size 0x8d(0x7d94 bytes) 2024-09-14T07:00:41.685 INFO:teuthology.orchestra.run.smithi007.stdout:5018 read 0x6d4b24f thru 0x6d58fa1 (0xdd53 bytes) 2024-09-14T07:00:41.686 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.686 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.686 INFO:teuthology.orchestra.run.smithi007.stdout:5021 punch from 0x73deea1 to 0x73e411f, (0x527e bytes) 2024-09-14T07:00:41.686 INFO:teuthology.orchestra.run.smithi007.stdout:5022 read 0x4c53584 thru 0x4c573c6 (0x3e43 bytes) 2024-09-14T07:00:41.686 INFO:teuthology.orchestra.run.smithi007.stdout:5023 trunc from 0xb6e0b45 to 0xc015018 2024-09-14T07:00:41.690 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:41.688+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:41.691 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.694 INFO:teuthology.orchestra.run.smithi007.stdout:5025 trunc from 0xc015018 to 0xe00b1ea 2024-09-14T07:00:41.708 INFO:teuthology.orchestra.run.smithi007.stdout:5026 trunc from 0xe00b1ea to 0x5b259be 2024-09-14T07:00:41.724 INFO:teuthology.orchestra.run.smithi007.stdout:5027 read 0x8a9ce0 thru 0x8abb39 (0x1e5a bytes) 2024-09-14T07:00:41.725 INFO:teuthology.orchestra.run.smithi007.stdout:5030 read 0x34c5bc5 thru 0x34c618b (0x5c7 bytes) 2024-09-14T07:00:41.725 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:41.725 INFO:teuthology.orchestra.run.smithi007.stdout:5033 read 0x17b4e51 thru 0x17bea5f (0x9c0f bytes) 2024-09-14T07:00:41.729 INFO:teuthology.orchestra.run.smithi007.stdout:5034 punch from 0x4e6798a to 0x4e74306, (0xc97c bytes) 2024-09-14T07:00:41.729 INFO:teuthology.orchestra.run.smithi007.stdout:5036 write 0x4c6408f thru 0x4c72245 (0xe1b7 bytes) 2024-09-14T07:00:41.738 INFO:teuthology.orchestra.run.smithi007.stdout:5037 writesame 0xccc479 thru 0xcd63b1 data_size 0x15(0x9f39 bytes) 2024-09-14T07:00:41.757 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:41.757 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:41.770 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:41.791 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:41.788+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:41.832 INFO:teuthology.orchestra.run.smithi007.stdout:5038 read 0x3bce8f4 thru 0x3bd09eb (0x20f8 bytes) 2024-09-14T07:00:41.832 INFO:teuthology.orchestra.run.smithi007.stdout:5039 read 0x3e1d683 thru 0x3e1de8c (0x80a bytes) 2024-09-14T07:00:41.838 INFO:teuthology.orchestra.run.smithi007.stdout:5041 writesame 0x62e8c3f thru 0x62f5c99 data_size 0xd(0xd05b bytes) 2024-09-14T07:00:41.844 INFO:teuthology.orchestra.run.smithi007.stdout:5043 trunc from 0x62f5c9a to 0xb180db9 2024-09-14T07:00:41.859 INFO:teuthology.orchestra.run.smithi007.stdout:5044 write 0xe7d20d7 thru 0xe7d4b32 (0x2a5c bytes) 2024-09-14T07:00:41.867 INFO:teuthology.orchestra.run.smithi007.stdout:5046 read 0xd9632c4 thru 0xd970f1e (0xdc5b bytes) 2024-09-14T07:00:41.867 INFO:teuthology.orchestra.run.smithi007.stdout:5047 write 0xbc53fc9 thru 0xbc5763d (0x3675 bytes) 2024-09-14T07:00:41.876 INFO:teuthology.orchestra.run.smithi007.stdout:5048 write 0x9a6fdf8 thru 0x9a7f810 (0xfa19 bytes) 2024-09-14T07:00:41.885 INFO:teuthology.orchestra.run.smithi007.stdout:5049 read 0xd371e70 thru 0xd37b388 (0x9519 bytes) 2024-09-14T07:00:41.885 INFO:teuthology.orchestra.run.smithi007.stdout:5050 write 0x1f00dcc thru 0x1f07d81 (0x6fb6 bytes) 2024-09-14T07:00:41.888 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:41.888 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:41.891 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:41.889+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:41.899 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:41.983 INFO:teuthology.orchestra.run.smithi007.stdout:5052 read 0x579169f thru 0x579ddf5 (0xc757 bytes) 2024-09-14T07:00:41.983 INFO:teuthology.orchestra.run.smithi007.stdout:5053 write 0x20e7f8 thru 0x216c69 (0x8472 bytes) 2024-09-14T07:00:41.992 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:41.991+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:42.032 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:42.032 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:42.042 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:42.065 INFO:teuthology.orchestra.run.smithi007.stdout:5056 trunc from 0xe7d4b33 to 0x5204634 2024-09-14T07:00:42.080 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.080 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.080 INFO:teuthology.orchestra.run.smithi007.stdout:5060 punch from 0x30c3178 to 0x30cc1b2, (0x903a bytes) 2024-09-14T07:00:42.080 INFO:teuthology.orchestra.run.smithi007.stdout:5061 read 0x647fa6 thru 0x64ccb5 (0x4d10 bytes) 2024-09-14T07:00:42.082 INFO:teuthology.orchestra.run.smithi007.stdout:5062 read 0x4753bf4 thru 0x4758699 (0x4aa6 bytes) 2024-09-14T07:00:42.082 INFO:teuthology.orchestra.run.smithi007.stdout:5063 read 0xaf2d19 thru 0xafc3cf (0x96b7 bytes) 2024-09-14T07:00:42.084 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.092 INFO:teuthology.orchestra.run.smithi007.stdout:5066 write 0x70662a1 thru 0x7067556 (0x12b6 bytes) 2024-09-14T07:00:42.096 INFO:teuthology.orchestra.run.smithi007.stdout:5067 writesame 0x21c1a9d thru 0x21ce096 data_size 0x3(0xc5fa bytes) 2024-09-14T07:00:42.131 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:42.130+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:42.193 INFO:teuthology.orchestra.run.smithi007.stdout:5068 writesame 0x3dfb22 thru 0x3ea184 data_size 0x7(0xa663 bytes) 2024-09-14T07:00:42.193 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:42.191+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:42.210 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:42.211 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:42.220 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:42.293 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:42.292+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:42.314 INFO:teuthology.orchestra.run.smithi007.stdout:5069 read 0x3009d1a thru 0x3014524 (0xa80b bytes) 2024-09-14T07:00:42.329 INFO:teuthology.orchestra.run.smithi007.stdout:5070 writesame 0xdbdcaa5 thru 0xdbdf85d data_size 0x1(0x2db9 bytes) 2024-09-14T07:00:42.329 INFO:teuthology.orchestra.run.smithi007.stdout:5071 read 0x53a4b7e thru 0x53a9e0e (0x5291 bytes) 2024-09-14T07:00:42.330 INFO:teuthology.orchestra.run.smithi007.stdout:5072 writesame 0x48b0aa0 thru 0x48bae6b data_size 0x16(0xa3cc bytes) 2024-09-14T07:00:42.336 INFO:teuthology.orchestra.run.smithi007.stdout:5074 write 0xc729f9b thru 0xc72f529 (0x558f bytes) 2024-09-14T07:00:42.343 INFO:teuthology.orchestra.run.smithi007.stdout:5075 write 0xa4f4e7e thru 0xa4f718f (0x2312 bytes) 2024-09-14T07:00:42.350 INFO:teuthology.orchestra.run.smithi007.stdout:5076 writesame 0x1c3972 thru 0x1d3061 data_size 0x30(0xf6f0 bytes) 2024-09-14T07:00:42.353 INFO:teuthology.orchestra.run.smithi007.stdout:5077 write 0xa7bc48e thru 0xa7bf636 (0x31a9 bytes) 2024-09-14T07:00:42.362 INFO:teuthology.orchestra.run.smithi007.stdout:5078 write 0x635060f thru 0x63549b4 (0x43a6 bytes) 2024-09-14T07:00:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:5080 read 0xb634b53 thru 0xb63c9e8 (0x7e96 bytes) 2024-09-14T07:00:42.370 INFO:teuthology.orchestra.run.smithi007.stdout:5081 write 0x6d4429e thru 0x6d4bba3 (0x7906 bytes) 2024-09-14T07:00:42.371 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:42.371 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:42.377 INFO:teuthology.orchestra.run.smithi007.stdout:5082 writesame 0x6254499 thru 0x6257a63 data_size 0x1(0x35cb bytes) 2024-09-14T07:00:42.381 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:42.385 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.385 INFO:teuthology.orchestra.run.smithi007.stdout:5084 read 0xa73b4e9 thru 0xa73c04d (0xb65 bytes) 2024-09-14T07:00:42.386 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.386 INFO:teuthology.orchestra.run.smithi007.stdout:5087 write 0x9d9f1ea thru 0x9da5b6b (0x6982 bytes) 2024-09-14T07:00:42.394 INFO:teuthology.orchestra.run.smithi007.stdout:5089 punch from 0x45ef788 to 0x45feca0, (0xf518 bytes) 2024-09-14T07:00:42.394 INFO:teuthology.orchestra.run.smithi007.stdout:5090 writesame 0x3364fac thru 0x3374a1c data_size 0x81(0xfa71 bytes) 2024-09-14T07:00:42.394 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:42.392+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:42.402 INFO:teuthology.orchestra.run.smithi007.stdout:5091 read 0x751632 thru 0x75b668 (0xa037 bytes) 2024-09-14T07:00:42.405 INFO:teuthology.orchestra.run.smithi007.stdout:5093 read 0x9ed5b37 thru 0x9ee2fe5 (0xd4af bytes) 2024-09-14T07:00:42.405 INFO:teuthology.orchestra.run.smithi007.stdout:5094 write 0x4e9833d thru 0x4ea16e5 (0x93a9 bytes) 2024-09-14T07:00:42.414 INFO:teuthology.orchestra.run.smithi007.stdout:5095 read 0x2488593 thru 0x2498168 (0xfbd6 bytes) 2024-09-14T07:00:42.420 INFO:teuthology.orchestra.run.smithi007.stdout:5096 write 0x93ea538 thru 0x93f7d77 (0xd840 bytes) 2024-09-14T07:00:42.429 INFO:teuthology.orchestra.run.smithi007.stdout:5097 write 0xb646ccd thru 0xb64fe73 (0x91a7 bytes) 2024-09-14T07:00:42.437 INFO:teuthology.orchestra.run.smithi007.stdout:5098 read 0xadfdb37 thru 0xae07133 (0x95fd bytes) 2024-09-14T07:00:42.437 INFO:teuthology.orchestra.run.smithi007.stdout:5099 punch from 0x45b31cb to 0x45b7187, (0x3fbc bytes) 2024-09-14T07:00:42.437 INFO:teuthology.orchestra.run.smithi007.stdout:5100 write 0xb3d0145 thru 0xb3d6e83 (0x6d3f bytes) 2024-09-14T07:00:42.446 INFO:teuthology.orchestra.run.smithi007.stdout:5101 read 0x8c6ee1d thru 0x8c7b64f (0xc833 bytes) 2024-09-14T07:00:42.446 INFO:teuthology.orchestra.run.smithi007.stdout:5102 writesame 0x679fbdc thru 0x67a361c data_size 0x3(0x3a41 bytes) 2024-09-14T07:00:42.454 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.463 INFO:teuthology.orchestra.run.smithi007.stdout:5108 trunc from 0xdbdf85e to 0x60b5e8d 2024-09-14T07:00:42.490 INFO:teuthology.orchestra.run.smithi007.stdout:5110 trunc from 0x60b5e8d to 0x3db5054 2024-09-14T07:00:42.494 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:42.493+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:42.497 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:42.497 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:42.506 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:42.509 INFO:teuthology.orchestra.run.smithi007.stdout:5111 trunc from 0x3db5054 to 0x5eee74 2024-09-14T07:00:42.559 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T07:00:42.596 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:42.595+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:42.631 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.639 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:42.639 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:42.648 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:42.652 INFO:teuthology.orchestra.run.smithi007.stdout:5114 writesame 0xd9f1e84 thru 0xda01d39 data_size 0x1(0xfeb6 bytes) 2024-09-14T07:00:42.666 INFO:teuthology.orchestra.run.smithi007.stdout:5115 trunc from 0xda01d3a to 0xd24415 2024-09-14T07:00:42.682 INFO:teuthology.orchestra.run.smithi007.stdout:5117 trunc from 0xd24415 to 0x9b94b7f 2024-09-14T07:00:42.688 INFO:teuthology.orchestra.run.smithi007.stdout:5118 read 0x35c62b3 thru 0x35c9fc3 (0x3d11 bytes) 2024-09-14T07:00:42.688 INFO:teuthology.orchestra.run.smithi007.stdout:5119 punch from 0x5f2824f to 0x5f35fc9, (0xdd7a bytes) 2024-09-14T07:00:42.688 INFO:teuthology.orchestra.run.smithi007.stdout:5120 trunc from 0x9b94b7f to 0x90a6fe1 2024-09-14T07:00:42.694 INFO:teuthology.orchestra.run.smithi007.stdout:5121 write 0x1e45f48 thru 0x1e54c6d (0xed26 bytes) 2024-09-14T07:00:42.702 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:42.702 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:42.702 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T07:00:42.702 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:00:42.702 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:00:42.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T07:00:42.703 INFO:teuthology.orchestra.run.smithi007.stdout:5122 read 0xe4e6e0 thru 0xe58788 (0xa0a9 bytes) 2024-09-14T07:00:42.703 INFO:teuthology.orchestra.run.smithi007.stdout:5123 read 0x7bb8992 thru 0x7bbf970 (0x6fdf bytes) 2024-09-14T07:00:42.703 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.703 INFO:teuthology.orchestra.run.smithi007.stdout:5126 punch from 0x2f02bee to 0x2f049a9, (0x1dbb bytes) 2024-09-14T07:00:42.704 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.704 INFO:teuthology.orchestra.run.smithi007.stdout:5129 writesame 0x3c66955 thru 0x3c69a5f data_size 0x9(0x310b bytes) 2024-09-14T07:00:42.715 INFO:teuthology.orchestra.run.smithi007.stdout:5130 trunc from 0x90a6fe1 to 0x56d1c68 2024-09-14T07:00:42.731 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:42.731+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:42.732 INFO:teuthology.orchestra.run.smithi007.stdout:5131 write 0xa02ed01 thru 0xa031be2 (0x2ee2 bytes) 2024-09-14T07:00:42.748 INFO:teuthology.orchestra.run.smithi007.stdout:5132 trunc from 0xa031be3 to 0x43155e1 2024-09-14T07:00:42.758 INFO:teuthology.orchestra.run.smithi007.stdout:5134 write 0x2c1620a thru 0x2c255c5 (0xf3bc bytes) 2024-09-14T07:00:42.766 INFO:teuthology.orchestra.run.smithi007.stdout:5135 read 0x304e908 thru 0x305d74f (0xee48 bytes) 2024-09-14T07:00:42.766 INFO:teuthology.orchestra.run.smithi007.stdout:5136 read 0x198200d thru 0x1984e7e (0x2e72 bytes) 2024-09-14T07:00:42.778 INFO:teuthology.orchestra.run.smithi007.stdout:5138 write 0xa2b7230 thru 0xa2be1a5 (0x6f76 bytes) 2024-09-14T07:00:42.788 INFO:teuthology.orchestra.run.smithi007.stdout:5139 writesame 0x28978f4 thru 0x289f2a4 data_size 0x1(0x79b1 bytes) 2024-09-14T07:00:42.797 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:42.796+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:42.798 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:42.798 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:42.801 INFO:teuthology.orchestra.run.smithi007.stdout:5140 write 0x4d7e2c6 thru 0x4d8ab92 (0xc8cd bytes) 2024-09-14T07:00:42.808 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:42.810 INFO:teuthology.orchestra.run.smithi007.stdout:5141 write 0x5e005ad thru 0x5e00cb3 (0x707 bytes) 2024-09-14T07:00:42.814 INFO:teuthology.orchestra.run.smithi007.stdout:5142 read 0x765bd1b thru 0x76689dd (0xccc3 bytes) 2024-09-14T07:00:42.814 INFO:teuthology.orchestra.run.smithi007.stdout:5144 read 0x5f90a39 thru 0x5f9ba6f (0xb037 bytes) 2024-09-14T07:00:42.815 INFO:teuthology.orchestra.run.smithi007.stdout:5146 read 0x28d6c84 thru 0x28dd5f9 (0x6976 bytes) 2024-09-14T07:00:42.826 INFO:teuthology.orchestra.run.smithi007.stdout:5147 write 0xec23ef8 thru 0xec32562 (0xe66b bytes) 2024-09-14T07:00:42.835 INFO:teuthology.orchestra.run.smithi007.stdout:5148 write 0x1594e74 thru 0x159d4c0 (0x864d bytes) 2024-09-14T07:00:42.843 INFO:teuthology.orchestra.run.smithi007.stdout:5149 read 0x5f7f515 thru 0x5f84f1b (0x5a07 bytes) 2024-09-14T07:00:42.843 INFO:teuthology.orchestra.run.smithi007.stdout:5150 read 0x1e145f7 thru 0x1e238aa (0xf2b4 bytes) 2024-09-14T07:00:42.844 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.844 INFO:teuthology.orchestra.run.smithi007.stdout:5154 write 0x8db86c8 thru 0x8dc6e5b (0xe794 bytes) 2024-09-14T07:00:42.844 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:42.844 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T07:00:42.844 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:42.853 INFO:teuthology.orchestra.run.smithi007.stdout:5155 read 0x9b1b072 thru 0x9b22f8e (0x7f1d bytes) 2024-09-14T07:00:42.853 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.853 INFO:teuthology.orchestra.run.smithi007.stdout:5158 writesame 0xbaf06d2 thru 0xbaf4d39 data_size 0x18(0x4668 bytes) 2024-09-14T07:00:42.861 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:42.861 INFO:teuthology.orchestra.run.smithi007.stdout:5160 write 0x55de34 thru 0x560d3b (0x2f08 bytes) 2024-09-14T07:00:42.863 INFO:teuthology.orchestra.run.smithi007.stdout:5161 read 0xe2ed085 thru 0xe2ef056 (0x1fd2 bytes) 2024-09-14T07:00:42.863 INFO:teuthology.orchestra.run.smithi007.stdout:5162 read 0x4cfb36b thru 0x4d03683 (0x8319 bytes) 2024-09-14T07:00:42.874 INFO:teuthology.orchestra.run.smithi007.stdout:5164 trunc from 0xec32563 to 0x5e5b403 2024-09-14T07:00:42.889 INFO:teuthology.orchestra.run.smithi007.stdout:5165 punch from 0x408ba95 to 0x408db60, (0x20cb bytes) 2024-09-14T07:00:42.897 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:42.897+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:42.903 INFO:teuthology.orchestra.run.smithi007.stdout:5166 write 0xd56fd25 thru 0xd5771f8 (0x74d4 bytes) 2024-09-14T07:00:42.916 INFO:teuthology.orchestra.run.smithi007.stdout:5167 write 0xdc62ba8 thru 0xdc724df (0xf938 bytes) 2024-09-14T07:00:42.923 INFO:teuthology.orchestra.run.smithi007.stdout:5168 write 0x69858e7 thru 0x698a83c (0x4f56 bytes) 2024-09-14T07:00:42.932 INFO:teuthology.orchestra.run.smithi007.stdout:5169 write 0x656e95b thru 0x65701a1 (0x1847 bytes) 2024-09-14T07:00:42.938 INFO:teuthology.orchestra.run.smithi007.stdout:5171 write 0x2e15922 thru 0x2e2325f (0xd93e bytes) 2024-09-14T07:00:42.946 INFO:teuthology.orchestra.run.smithi007.stdout:5172 punch from 0x5840c62 to 0x5844216, (0x35b4 bytes) 2024-09-14T07:00:42.961 INFO:teuthology.orchestra.run.smithi007.stdout:5173 trunc from 0xdc724e0 to 0xf7f061 2024-09-14T07:00:42.961 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:42.961 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:42.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:42.999 INFO:teuthology.orchestra.run.smithi007.stdout:5174 writesame 0x8204e4d thru 0x8212bb1 data_size 0x13(0xdd65 bytes) 2024-09-14T07:00:42.999 INFO:teuthology.orchestra.run.smithi007.stdout:5177 read 0x4a2cbf2 thru 0x4a375c8 (0xa9d7 bytes) 2024-09-14T07:00:42.999 INFO:teuthology.orchestra.run.smithi007.stdout:5178 write 0x15e25f6 thru 0x15edcf6 (0xb701 bytes) 2024-09-14T07:00:42.999 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:42.998+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:43.017 INFO:teuthology.orchestra.run.smithi007.stdout:5183 write 0xae96872 thru 0xaea011d (0x98ac bytes) 2024-09-14T07:00:43.026 INFO:teuthology.orchestra.run.smithi007.stdout:5184 read 0x6962e4a thru 0x6968641 (0x57f8 bytes) 2024-09-14T07:00:43.026 INFO:teuthology.orchestra.run.smithi007.stdout:5185 punch from 0x835e998 to 0x8365d01, (0x7369 bytes) 2024-09-14T07:00:43.026 INFO:teuthology.orchestra.run.smithi007.stdout:5186 punch from 0x8ddae85 to 0x8de52c2, (0xa43d bytes) 2024-09-14T07:00:43.026 INFO:teuthology.orchestra.run.smithi007.stdout:5187 punch from 0x6313bef to 0x6320d6d, (0xd17e bytes) 2024-09-14T07:00:43.038 INFO:teuthology.orchestra.run.smithi007.stdout:5188 trunc from 0xaea011e to 0xea44a 2024-09-14T07:00:43.098 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:43.098+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:43.114 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:43.114 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:43.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:43.196 INFO:teuthology.orchestra.run.smithi007.stdout:5190 writesame 0xd7614 thru 0xe2010 data_size 0x1(0xa9fd bytes) 2024-09-14T07:00:43.200 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:43.199+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:43.221 INFO:teuthology.orchestra.run.smithi007.stdout:5191 trunc from 0xea44a to 0xed970a8 2024-09-14T07:00:43.227 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.227 INFO:teuthology.orchestra.run.smithi007.stdout:5193 read 0xb804d29 thru 0xb81175a (0xca32 bytes) 2024-09-14T07:00:43.230 INFO:teuthology.orchestra.run.smithi007.stdout:5194 trunc from 0xed970a8 to 0xcb4fa47 2024-09-14T07:00:43.235 INFO:teuthology.orchestra.run.smithi007.stdout:5195 read 0x2e908 thru 0x3738c (0x8a85 bytes) 2024-09-14T07:00:43.236 INFO:teuthology.orchestra.run.smithi007.stdout:5196 punch from 0xa575eb8 to 0xa5818dd, (0xba25 bytes) 2024-09-14T07:00:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:5198 read 0x40b959d thru 0x40bdffd (0x4a61 bytes) 2024-09-14T07:00:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:5200 read 0x99cc230 thru 0x99d5469 (0x923a bytes) 2024-09-14T07:00:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:5201 read 0x437f314 thru 0x4385fa1 (0x6c8e bytes) 2024-09-14T07:00:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:5202 punch from 0x5cd0acc to 0x5cdc1c5, (0xb6f9 bytes) 2024-09-14T07:00:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:5203 read 0x83377e0 thru 0x833cd16 (0x5537 bytes) 2024-09-14T07:00:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:5204 read 0x382b526 thru 0x382fc86 (0x4761 bytes) 2024-09-14T07:00:43.237 INFO:teuthology.orchestra.run.smithi007.stdout:5205 trunc from 0xcb4fa47 to 0xd4ec3bf 2024-09-14T07:00:43.242 INFO:teuthology.orchestra.run.smithi007.stdout:5206 read 0x1612ee2 thru 0x161d3e4 (0xa503 bytes) 2024-09-14T07:00:43.246 INFO:teuthology.orchestra.run.smithi007.stdout:5207 trunc from 0xd4ec3bf to 0xa63e86a 2024-09-14T07:00:43.254 INFO:teuthology.orchestra.run.smithi007.stdout:5208 trunc from 0xa63e86a to 0x7e83e38 2024-09-14T07:00:43.260 INFO:teuthology.orchestra.run.smithi007.stdout:5209 writesame 0x3fc83e8 thru 0x3fcec71 data_size 0x1(0x688a bytes) 2024-09-14T07:00:43.280 INFO:teuthology.orchestra.run.smithi007.stdout:5212 writesame 0xb3d9a67 thru 0xb3dffc9 data_size 0x1d(0x6563 bytes) 2024-09-14T07:00:43.281 INFO:teuthology.orchestra.run.smithi007.stdout:5214 writesame 0x20d1fbd thru 0x20da4fd data_size 0x53(0x8541 bytes) 2024-09-14T07:00:43.290 INFO:teuthology.orchestra.run.smithi007.stdout:5218 read 0x2426e7b thru 0x2430cd0 (0x9e56 bytes) 2024-09-14T07:00:43.290 INFO:teuthology.orchestra.run.smithi007.stdout:5219 punch from 0x7931b33 to 0x793d37c, (0xb849 bytes) 2024-09-14T07:00:43.290 INFO:teuthology.orchestra.run.smithi007.stdout:5220 read 0x15452c6 thru 0x1547443 (0x217e bytes) 2024-09-14T07:00:43.291 INFO:teuthology.orchestra.run.smithi007.stdout:5221 write 0x7e9046e thru 0x7e96434 (0x5fc7 bytes) 2024-09-14T07:00:43.299 INFO:teuthology.orchestra.run.smithi007.stdout:5223 punch from 0x53169e4 to 0x5319fe2, (0x35fe bytes) 2024-09-14T07:00:43.299 INFO:teuthology.orchestra.run.smithi007.stdout:5224 write 0xa38c1a2 thru 0xa38db36 (0x1995 bytes) 2024-09-14T07:00:43.300 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:43.299+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:43.300 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:43.301 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:43.309 INFO:teuthology.orchestra.run.smithi007.stdout:5225 trunc from 0xb3dffca to 0x7986060 2024-09-14T07:00:43.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:43.332 INFO:teuthology.orchestra.run.smithi007.stdout:5226 writesame 0xd393a21 thru 0xd39bf7b data_size 0x1(0x855b bytes) 2024-09-14T07:00:43.332 INFO:teuthology.orchestra.run.smithi007.stdout:5228 punch from 0x3c5a28f to 0x3c5f9d6, (0x5747 bytes) 2024-09-14T07:00:43.335 INFO:teuthology.orchestra.run.smithi007.stdout:5229 trunc from 0xd39bf7c to 0xacea3da 2024-09-14T07:00:43.341 INFO:teuthology.orchestra.run.smithi007.stdout:5230 punch from 0x70c0150 to 0x70cdbca, (0xda7a bytes) 2024-09-14T07:00:43.341 INFO:teuthology.orchestra.run.smithi007.stdout:5231 read 0xaca605e thru 0xacb2325 (0xc2c8 bytes) 2024-09-14T07:00:43.349 INFO:teuthology.orchestra.run.smithi007.stdout:5232 writesame 0xc43516a thru 0xc43e662 data_size 0x1(0x94f9 bytes) 2024-09-14T07:00:43.349 INFO:teuthology.orchestra.run.smithi007.stdout:5233 read 0x46db89a thru 0x46e53c7 (0x9b2e bytes) 2024-09-14T07:00:43.349 INFO:teuthology.orchestra.run.smithi007.stdout:5235 read 0x64fa8eb thru 0x6506e8b (0xc5a1 bytes) 2024-09-14T07:00:43.349 INFO:teuthology.orchestra.run.smithi007.stdout:5236 read 0xa8d1caf thru 0xa8dcf58 (0xb2aa bytes) 2024-09-14T07:00:43.357 INFO:teuthology.orchestra.run.smithi007.stdout:5237 writesame 0xd9ac5d1 thru 0xd9afe30 data_size 0x52(0x3860 bytes) 2024-09-14T07:00:43.362 INFO:teuthology.orchestra.run.smithi007.stdout:5238 writesame 0xe4cc970 thru 0xe4d08ae data_size 0x15(0x3f3f bytes) 2024-09-14T07:00:43.362 INFO:teuthology.orchestra.run.smithi007.stdout:5239 write 0xa25c198 thru 0xa2606ac (0x4515 bytes) 2024-09-14T07:00:43.375 INFO:teuthology.orchestra.run.smithi007.stdout:5241 trunc from 0xe4d08af to 0xa4bd542 2024-09-14T07:00:43.388 INFO:teuthology.orchestra.run.smithi007.stdout:5243 writesame 0xc25c52d thru 0xc268e4c data_size 0x10(0xc920 bytes) 2024-09-14T07:00:43.388 INFO:teuthology.orchestra.run.smithi007.stdout:5245 read 0x55580c6 thru 0x555bb9d (0x3ad8 bytes) 2024-09-14T07:00:43.388 INFO:teuthology.orchestra.run.smithi007.stdout:5246 write 0x2aaa430 thru 0x2aac8dc (0x24ad bytes) 2024-09-14T07:00:43.396 INFO:teuthology.orchestra.run.smithi007.stdout:5247 writesame 0x76ca591 thru 0x76d81ad data_size 0x9(0xdc1d bytes) 2024-09-14T07:00:43.400 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:43.400+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:43.406 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.406 INFO:teuthology.orchestra.run.smithi007.stdout:5249 writesame 0x3a8cdea thru 0x3a8f6fc data_size 0x5(0x2913 bytes) 2024-09-14T07:00:43.415 INFO:teuthology.orchestra.run.smithi007.stdout:5250 read 0x97ba188 thru 0x97c8fea (0xee63 bytes) 2024-09-14T07:00:43.416 INFO:teuthology.orchestra.run.smithi007.stdout:5251 trunc from 0xc268e4d to 0xd4b3292 2024-09-14T07:00:43.422 INFO:teuthology.orchestra.run.smithi007.stdout:5253 write 0x3a3bd4a thru 0x3a45cbe (0x9f75 bytes) 2024-09-14T07:00:43.431 INFO:teuthology.orchestra.run.smithi007.stdout:5255 writesame 0x1430037 thru 0x143aa4c data_size 0x7b(0xaa16 bytes) 2024-09-14T07:00:43.449 INFO:teuthology.orchestra.run.smithi007.stdout:5257 trunc from 0xd4b3292 to 0x4633fd2 2024-09-14T07:00:43.461 INFO:teuthology.orchestra.run.smithi007.stdout:5258 punch from 0x183e0d9 to 0x183fa4b, (0x1972 bytes) 2024-09-14T07:00:43.467 INFO:teuthology.orchestra.run.smithi007.stdout:5259 writesame 0x4e773cd thru 0x4e78923 data_size 0x3(0x1557 bytes) 2024-09-14T07:00:43.467 INFO:teuthology.orchestra.run.smithi007.stdout:5260 read 0x28b287e thru 0x28b65d6 (0x3d59 bytes) 2024-09-14T07:00:43.468 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:43.468 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:43.477 INFO:teuthology.orchestra.run.smithi007.stdout:5261 trunc from 0x4e78924 to 0xd4167c7 2024-09-14T07:00:43.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:43.483 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.483 INFO:teuthology.orchestra.run.smithi007.stdout:5265 punch from 0x6f91cf3 to 0x6f94d62, (0x306f bytes) 2024-09-14T07:00:43.483 INFO:teuthology.orchestra.run.smithi007.stdout:5268 writesame 0xaded6d9 thru 0xadf6bca data_size 0x7b(0x94f2 bytes) 2024-09-14T07:00:43.491 INFO:teuthology.orchestra.run.smithi007.stdout:5269 write 0x8c07c09 thru 0x8c12364 (0xa75c bytes) 2024-09-14T07:00:43.500 INFO:teuthology.orchestra.run.smithi007.stdout:5270 writesame 0x464b8a thru 0x467382 data_size 0x9(0x27f9 bytes) 2024-09-14T07:00:43.501 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:43.500+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:43.510 INFO:teuthology.orchestra.run.smithi007.stdout:5271 write 0xd23ee7 thru 0xd30180 (0xc29a bytes) 2024-09-14T07:00:43.520 INFO:teuthology.orchestra.run.smithi007.stdout:5272 writesame 0x2a527a4 thru 0x2a5bc05 data_size 0x27(0x9462 bytes) 2024-09-14T07:00:43.523 INFO:teuthology.orchestra.run.smithi007.stdout:5273 read 0xa0c57a7 thru 0xa0c8989 (0x31e3 bytes) 2024-09-14T07:00:43.523 INFO:teuthology.orchestra.run.smithi007.stdout:5274 punch from 0x8c44f1 to 0x8ce590, (0xa09f bytes) 2024-09-14T07:00:43.523 INFO:teuthology.orchestra.run.smithi007.stdout:5275 write 0x75765b9 thru 0x7585b35 (0xf57d bytes) 2024-09-14T07:00:43.532 INFO:teuthology.orchestra.run.smithi007.stdout:5277 write 0x26d0e57 thru 0x26d4417 (0x35c1 bytes) 2024-09-14T07:00:43.541 INFO:teuthology.orchestra.run.smithi007.stdout:5279 trunc from 0xd4167c7 to 0xd607bed 2024-09-14T07:00:43.547 INFO:teuthology.orchestra.run.smithi007.stdout:5281 write 0x256bc33 thru 0x256bc38 (0x6 bytes) 2024-09-14T07:00:43.551 INFO:teuthology.orchestra.run.smithi007.stdout:5282 write 0x419f4a6 thru 0x41a89a8 (0x9503 bytes) 2024-09-14T07:00:43.561 INFO:teuthology.orchestra.run.smithi007.stdout:5283 write 0xb775ed1 thru 0xb7781aa (0x22da bytes) 2024-09-14T07:00:43.570 INFO:teuthology.orchestra.run.smithi007.stdout:5285 writesame 0x784890b thru 0x7853c56 data_size 0x44(0xb34c bytes) 2024-09-14T07:00:43.579 INFO:teuthology.orchestra.run.smithi007.stdout:5287 write 0x58967ba thru 0x58a1acf (0xb316 bytes) 2024-09-14T07:00:43.589 INFO:teuthology.orchestra.run.smithi007.stdout:5288 read 0x508f284 thru 0x508f307 (0x84 bytes) 2024-09-14T07:00:43.589 INFO:teuthology.orchestra.run.smithi007.stdout:5289 write 0x275d57f thru 0x27615db (0x405d bytes) 2024-09-14T07:00:43.597 INFO:teuthology.orchestra.run.smithi007.stdout:5290 writesame 0xc7e7e45 thru 0xc7f61c5 data_size 0x1(0xe381 bytes) 2024-09-14T07:00:43.601 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:43.601+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:43.611 INFO:teuthology.orchestra.run.smithi007.stdout:5291 write 0xb89684a thru 0xb89d6cb (0x6e82 bytes) 2024-09-14T07:00:43.621 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.621 INFO:teuthology.orchestra.run.smithi007.stdout:5296 writesame 0x82ac7b4 thru 0x82b952f data_size 0x2(0xcd7c bytes) 2024-09-14T07:00:43.633 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:43.633 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:43.638 INFO:teuthology.orchestra.run.smithi007.stdout:5297 writesame 0xdd6a91e thru 0xdd79b87 data_size 0x3(0xf26a bytes) 2024-09-14T07:00:43.638 INFO:teuthology.orchestra.run.smithi007.stdout:5298 read 0x72b537c thru 0x72c4e5c (0xfae1 bytes) 2024-09-14T07:00:43.638 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.638 INFO:teuthology.orchestra.run.smithi007.stdout:5303 write 0xbb4266e thru 0xbb4d6f3 (0xb086 bytes) 2024-09-14T07:00:43.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:43.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:43.647 INFO:teuthology.orchestra.run.smithi007.stdout:5304 read 0xb78d499 thru 0xb793c71 (0x67d9 bytes) 2024-09-14T07:00:43.648 INFO:teuthology.orchestra.run.smithi007.stdout:5305 writesame 0x4a29aa1 thru 0x4a30ffa data_size 0x6(0x755a bytes) 2024-09-14T07:00:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:5306 writesame 0xeafe413 thru 0xeb0356e data_size 0x52(0x515c bytes) 2024-09-14T07:00:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:5307 punch from 0x150e768 to 0x1511577, (0x2e0f bytes) 2024-09-14T07:00:43.666 INFO:teuthology.orchestra.run.smithi007.stdout:5308 write 0x1b54c18 thru 0x1b58065 (0x344e bytes) 2024-09-14T07:00:43.676 INFO:teuthology.orchestra.run.smithi007.stdout:5309 writesame 0xadf86df thru 0xae0039e data_size 0x20(0x7cc0 bytes) 2024-09-14T07:00:43.684 INFO:teuthology.orchestra.run.smithi007.stdout:5312 read 0x37ef1de thru 0x37f681f (0x7642 bytes) 2024-09-14T07:00:43.684 INFO:teuthology.orchestra.run.smithi007.stdout:5313 read 0x86db41c thru 0x86e77ad (0xc392 bytes) 2024-09-14T07:00:43.684 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.684 INFO:teuthology.orchestra.run.smithi007.stdout:5315 read 0xa4d54bf thru 0xa4e3dfb (0xe93d bytes) 2024-09-14T07:00:43.684 INFO:teuthology.orchestra.run.smithi007.stdout:5316 read 0xd2f3098 thru 0xd302f06 (0xfe6f bytes) 2024-09-14T07:00:43.684 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.685 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.685 INFO:teuthology.orchestra.run.smithi007.stdout:5319 punch from 0xb7c9596 to 0xb7d1831, (0x829b bytes) 2024-09-14T07:00:43.685 INFO:teuthology.orchestra.run.smithi007.stdout:5320 read 0xa15bd54 thru 0xa15dfc9 (0x2276 bytes) 2024-09-14T07:00:43.685 INFO:teuthology.orchestra.run.smithi007.stdout:5321 write 0x90a86fe thru 0x90accae (0x45b1 bytes) 2024-09-14T07:00:43.702 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:43.701+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:43.764 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:43.766 INFO:teuthology.orchestra.run.smithi007.stdout:5322 punch from 0xe011ef1 to 0xe0151aa, (0x32b9 bytes) 2024-09-14T07:00:43.772 INFO:teuthology.orchestra.run.smithi007.stdout:5324 write 0xed8be17 thru 0xed9bc44 (0xfe2e bytes) 2024-09-14T07:00:43.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:43.782 INFO:teuthology.orchestra.run.smithi007.stdout:5326 read 0x3909922 thru 0x39172b4 (0xd993 bytes) 2024-09-14T07:00:43.783 INFO:teuthology.orchestra.run.smithi007.stdout:5327 writesame 0xc050649 thru 0xc0555aa data_size 0x6(0x4f62 bytes) 2024-09-14T07:00:43.794 INFO:teuthology.orchestra.run.smithi007.stdout:5328 punch from 0x7ede62d to 0x7ee2065, (0x3a38 bytes) 2024-09-14T07:00:43.794 INFO:teuthology.orchestra.run.smithi007.stdout:5329 writesame 0x5a027f7 thru 0x5a0dfa7 data_size 0xab(0xb7b1 bytes) 2024-09-14T07:00:43.803 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:43.802+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:43.804 INFO:teuthology.orchestra.run.smithi007.stdout:5331 write 0x84f78ef thru 0x85032bb (0xb9cd bytes) 2024-09-14T07:00:43.804 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:43.805 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:43.814 INFO:teuthology.orchestra.run.smithi007.stdout:5332 writesame 0xdb8fee4 thru 0xdb944a1 data_size 0x2(0x45be bytes) 2024-09-14T07:00:43.825 INFO:teuthology.orchestra.run.smithi007.stdout:5333 writesame 0x2240ef4 thru 0x22482bd data_size 0x1(0x73ca bytes) 2024-09-14T07:00:43.834 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:43.834 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:43.840 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.840 INFO:teuthology.orchestra.run.smithi007.stdout:5335 punch from 0x36135f3 to 0x3617349, (0x3d56 bytes) 2024-09-14T07:00:43.840 INFO:teuthology.orchestra.run.smithi007.stdout:5337 write 0x484197a thru 0x4844e95 (0x351c bytes) 2024-09-14T07:00:43.849 INFO:teuthology.orchestra.run.smithi007.stdout:5338 writesame 0xd1b81cc thru 0xd1c6e55 data_size 0x2(0xec8a bytes) 2024-09-14T07:00:43.859 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:43.868 INFO:teuthology.orchestra.run.smithi007.stdout:5339 trunc from 0xed9bc45 to 0x8ba83b6 2024-09-14T07:00:43.887 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:43.896 INFO:teuthology.orchestra.run.smithi007.stdout:5340 read 0x305a769 thru 0x3068c38 (0xe4d0 bytes) 2024-09-14T07:00:43.904 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:43.903+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:43.907 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:43.907 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:43.907 INFO:teuthology.orchestra.run.smithi007.stdout:5342 write 0xc6c4267 thru 0xc6ce0c1 (0x9e5b bytes) 2024-09-14T07:00:43.917 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.917 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:43.926 INFO:teuthology.orchestra.run.smithi007.stdout:5345 trunc from 0xc6ce0c2 to 0x58c26c8 2024-09-14T07:00:43.934 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:00:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.198008883, 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:00:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:00:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:17618 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.695395945000001, 2024-09-14T07:00:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016649290000000001, 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17618 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236329+0000", 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.236332+0000", 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237113+0000", 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078082900000000003 2024-09-14T07:00:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237131+0000", 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7747000000000001e-05 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237141+0000", 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0137e-05 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237221+0000", 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0482000000000006e-05 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237383+0000", 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016176100000000001 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.237737+0000", 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035380999999999998 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.241638+0000", 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039011739999999999 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252854+0000", 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011216097 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252919+0000", 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5129999999999995e-05 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:44.252979+0000", 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9573e-05 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40284 3.10 3:0c7b1f3b:::rbd_data.114f54fbf0ed.0000000000000004:9 [read 319488~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.727268883999997, 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016005640000000002, 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40284 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204456+0000", 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204457+0000", 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204459+0000", 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204466+0000", 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8730000000000001e-06 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.204472+0000", 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4779999999999999e-06 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207358+0000", 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028862549999999999 2024-09-14T07:00:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.207430+0000", 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2268000000000007e-05 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.220462+0000", 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013031951999999999 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40646 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 778240~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.726127249999998, 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017879991000000001, 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40646 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205598+0000", 2024-09-14T07:00:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205605+0000", 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2260000000000002e-06 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205611+0000", 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3289999999999999e-06 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205622+0000", 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213726+0000", 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.008104745 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.213792+0000", 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5401999999999995e-05 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223478+0000", 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096862100000000007 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40683 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 671744~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.726055330999998, 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016494009, 2024-09-14T07:00:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40683 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205670+0000", 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205672+0000", 2024-09-14T07:00:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0379999999999998e-06 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205679+0000", 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205684+0000", 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9359999999999998e-06 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221845+0000", 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016161422000000002 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.221913+0000", 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7669000000000006e-05 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222164+0000", 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025130600000000002 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40685 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.726034927000001, 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016657634000000001, 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40685 2024-09-14T07:00:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205690+0000", 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205691+0000", 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205693+0000", 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9e-06 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205699+0000", 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205703+0000", 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5240000000000002e-06 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222190+0000", 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016486752 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222257+0000", 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7095999999999996e-05 2024-09-14T07:00:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222348+0000", 2024-09-14T07:00:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1061000000000001e-05 2024-09-14T07:00:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40687 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 819200~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.726015697000001, 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016826543999999999, 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40687 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205710+0000", 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4999999999999998e-07 2024-09-14T07:00:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205712+0000", 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205718+0000", 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205723+0000", 2024-09-14T07:00:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222374+0000", 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016650933999999999 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222445+0000", 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1304999999999996e-05 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222536+0000", 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1104000000000001e-05 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40720 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 368640~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.725899101000003, 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016893371000000001, 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40720 2024-09-14T07:00:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205826+0000", 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205827+0000", 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.58e-07 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205834+0000", 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2729999999999997e-06 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205841+0000", 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8260000000000004e-06 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205846+0000", 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1769999999999997e-06 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222562+0000", 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016716178000000002 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222633+0000", 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0461000000000002e-05 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222720+0000", 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6997999999999996e-05 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40721 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 442368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.725871398999999, 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044822000000001, 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40721 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205854+0000", 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205857+0000", 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1639999999999999e-06 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205863+0000", 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205868+0000", 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0000000000000004e-06 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222745+0000", 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016877293000000002 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222814+0000", 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8806000000000005e-05 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222899+0000", 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4913000000000005e-05 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40742 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.725850833000003, 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017201659000000001, 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40742 2024-09-14T07:00:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205875+0000", 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205892+0000", 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7008e-05 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205899+0000", 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1489999999999997e-06 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205904+0000", 2024-09-14T07:00:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-14T07:00:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222923+0000", 2024-09-14T07:00:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017018229999999999 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.222991+0000", 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8545000000000002e-05 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223076+0000", 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4938999999999994e-05 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:40743 3.1a 3:5b73d5af:::rbd_data.114f54fbf0ed.0000000000000006:9 [read 811008~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.725812101000002, 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343654999999999, 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 40743 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205913+0000", 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205914+0000", 2024-09-14T07:00:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-07 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205916+0000", 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0559999999999999e-06 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205922+0000", 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3659999999999997e-06 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.205927+0000", 2024-09-14T07:00:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8169999999999998e-06 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223101+0000", 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017174452 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223168+0000", 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6756999999999998e-05 2024-09-14T07:00:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:48.223257+0000", 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8700000000000001e-05 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43579 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 107520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.475863472, 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01522916, 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43579 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455862+0000", 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455864+0000", 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455870+0000", 2024-09-14T07:00:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455875+0000", 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9509999999999997e-06 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470879+0000", 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015003729 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.470960+0000", 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1228000000000002e-05 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471091+0000", 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000131018 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43580 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 110592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.475843933999997, 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015447047, 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43580 2024-09-14T07:00:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455881+0000", 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455882+0000", 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5100000000000001e-07 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455884+0000", 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2010000000000002e-06 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455891+0000", 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5690000000000001e-06 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455896+0000", 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3639999999999998e-06 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471123+0000", 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015226805 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471200+0000", 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7405999999999997e-05 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471328+0000", 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012835099999999999 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:43581 3.1a 3:5c520e90:::rbd_data.114fb7f753fa.000000000000003c:a [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.475822033, 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015668949000000001, 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 43581 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455903+0000", 2024-09-14T07:00:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455904+0000", 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455910+0000", 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6619999999999999e-06 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455917+0000", 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5769999999999999e-06 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.455923+0000", 2024-09-14T07:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6779999999999997e-06 2024-09-14T07:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471360+0000", 2024-09-14T07:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015436854999999999 2024-09-14T07:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471443+0000", 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3815999999999998e-05 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.471572+0000", 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012892900000000001 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117541 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.040120098000003, 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015320307999999999, 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117541 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891605+0000", 2024-09-14T07:00:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891606+0000", 2024-09-14T07:00:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-14T07:00:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891609+0000", 2024-09-14T07:00:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.886e-06 2024-09-14T07:00:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891615+0000", 2024-09-14T07:00:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5010000000000001e-06 2024-09-14T07:00:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891620+0000", 2024-09-14T07:00:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901891+0000", 2024-09-14T07:00:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010270833 2024-09-14T07:00:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.901942+0000", 2024-09-14T07:00:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0612000000000003e-05 2024-09-14T07:00:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902142+0000", 2024-09-14T07:00:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020015500000000001 2024-09-14T07:00:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904269+0000", 2024-09-14T07:00:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021265519999999999 2024-09-14T07:00:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906874+0000", 2024-09-14T07:00:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026050800000000001 2024-09-14T07:00:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906904+0000", 2024-09-14T07:00:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199e-05 2024-09-14T07:00:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906926+0000", 2024-09-14T07:00:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1708e-05 2024-09-14T07:00:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117542 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.040098399000001, 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015409499, 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117542 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891627+0000", 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891630+0000", 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8569999999999999e-06 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891638+0000", 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5610000000000001e-06 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891643+0000", 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4299999999999997e-06 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902805+0000", 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011162071000000001 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.902879+0000", 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4152000000000004e-05 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903129+0000", 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024969499999999997 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904277+0000", 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001147906 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906968+0000", 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002690942 2024-09-14T07:00:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.906999+0000", 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1056000000000002e-05 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.907036+0000", 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7410000000000003e-05 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:117543 3.1a 3:5e78b169:::rbd_object_map.114f74d20483:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.040065939000002, 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018002384999999999, 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 117543 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891659+0000", 2024-09-14T07:00:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891660+0000", 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8100000000000003e-07 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891663+0000", 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2739999999999999e-06 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891671+0000", 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.303e-06 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.891678+0000", 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3709999999999996e-06 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903501+0000", 2024-09-14T07:00:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011823382 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903577+0000", 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5947999999999999e-05 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.903855+0000", 2024-09-14T07:00:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00027763899999999999 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.904798+0000", 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00094325599999999996 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909603+0000", 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048047539999999996 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909634+0000", 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1137999999999999e-05 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.909662+0000", 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7838999999999999e-05 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:120396 3.1c 3:3ebb3b40:::rbd_object_map.114f74d20483.0000000000000015:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.373101222999999, 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015664009999999999, 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 120396 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558624+0000", 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558625+0000", 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5099999999999998e-07 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558631+0000", 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558647+0000", 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5756999999999998e-05 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558657+0000", 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.995e-06 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558724+0000", 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7008000000000006e-05 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.558815+0000", 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1077999999999998e-05 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.559173+0000", 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035768600000000002 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560127+0000", 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095468799999999998 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574168+0000", 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014040238999999999 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574233+0000", 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4999999999999994e-05 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.574288+0000", 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.558e-05 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:134959 3.1a 3:59279a6c:::rbd_data.114fdc2d0201.0000000000000072:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:21.970619+0000", 2024-09-14T07:00:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.961106569999998, 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021043874000000001, 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 134959 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970619+0000", 2024-09-14T07:00:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970619+0000", 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970620+0000", 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970625+0000", 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5450000000000003e-06 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970638+0000", 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2777e-05 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970646+0000", 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9680000000000006e-06 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970714+0000", 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8361999999999994e-05 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.970796+0000", 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1577999999999997e-05 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.974492+0000", 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036963090000000001 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991537+0000", 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017044678000000001 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991586+0000", 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8689000000000001e-05 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991627+0000", 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1085999999999999e-05 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:21.991663+0000", 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5936999999999998e-05 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138654 3.1c 3:3ed33fa1:::rbd_data.114f74bd3bbf.00000000000000cc:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78)", 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:31.531314+0000", 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.400411137000001, 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016015504, 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138654 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531314+0000", 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531314+0000", 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531315+0000", 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000004e-07 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531319+0000", 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1999999999999996e-06 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531330+0000", 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0414999999999999e-05 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531336+0000", 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2389999999999999e-06 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531382+0000", 2024-09-14T07:00:44.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5867e-05 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.531474+0000", 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2489000000000002e-05 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.534588+0000", 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031140849999999999 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:44.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547207+0000", 2024-09-14T07:00:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012618994999999999 2024-09-14T07:00:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547261+0000", 2024-09-14T07:00:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3269000000000001e-05 2024-09-14T07:00:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547298+0000", 2024-09-14T07:00:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7471000000000001e-05 2024-09-14T07:00:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.547330+0000", 2024-09-14T07:00:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1680000000000002e-05 2024-09-14T07:00:44.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.050 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:44.051 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:44.051 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:44.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:5346 trunc from 0x58c26c8 to 0xc23d4e4 2024-09-14T07:00:44.053 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:44.053 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:5347 punch from 0x22e99cb to 0x22f5ff1, (0xc626 bytes) 2024-09-14T07:00:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:5349 read 0x7128ee1 thru 0x71365cd (0xd6ed bytes) 2024-09-14T07:00:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:5350 writesame 0x74c32bf thru 0x74d28fb data_size 0xd(0xf63d bytes) 2024-09-14T07:00:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.054 INFO:teuthology.orchestra.run.smithi007.stdout:5354 write 0x4c07add thru 0x4c09b2d (0x2051 bytes) 2024-09-14T07:00:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:5355 read 0x95bbced thru 0x95c0989 (0x4c9d bytes) 2024-09-14T07:00:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:5356 read 0x3d31eea thru 0x3d3bec1 (0x9fd8 bytes) 2024-09-14T07:00:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:44.055 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:5358 write 0xd1f4601 thru 0xd1fb110 (0x6b10 bytes) 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:5361 read 0x17c3532 thru 0x17c537d (0x1e4c bytes) 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:5363 read 0x93d4f84 thru 0x93e0291 (0xb30e bytes) 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:5365 read 0x3a6c7f0 thru 0x3a7c75e (0xff6f bytes) 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:5367 write 0x137e47d thru 0x138b0ff (0xcc83 bytes) 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:5374 write 0xc1e0725 thru 0xc1ec27f (0xbb5b bytes) 2024-09-14T07:00:44.057 INFO:teuthology.orchestra.run.smithi007.stdout:5376 trunc from 0xd1fb111 to 0x270a9cc 2024-09-14T07:00:44.058 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:44.050+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:44.073 INFO:teuthology.orchestra.run.smithi007.stdout:5377 writesame 0xeb7bd8f thru 0xeb7e746 data_size 0x3c(0x29b8 bytes) 2024-09-14T07:00:44.073 INFO:teuthology.orchestra.run.smithi007.stdout:5378 writesame 0x8ec76bc thru 0x8ed56fa data_size 0x3b(0xe03f bytes) 2024-09-14T07:00:44.083 INFO:teuthology.orchestra.run.smithi007.stdout:5379 write 0x9022e72 thru 0x902deae (0xb03d bytes) 2024-09-14T07:00:44.091 INFO:teuthology.orchestra.run.smithi007.stdout:5380 write 0x7fb70ab thru 0x7fbcc70 (0x5bc6 bytes) 2024-09-14T07:00:44.101 INFO:teuthology.orchestra.run.smithi007.stdout:5381 writesame 0x5e44cef thru 0x5e45b4a data_size 0x1(0xe5c bytes) 2024-09-14T07:00:44.107 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.107 INFO:teuthology.orchestra.run.smithi007.stdout:5385 read 0xa21ebb6 thru 0xa225e11 (0x725c bytes) 2024-09-14T07:00:44.107 INFO:teuthology.orchestra.run.smithi007.stdout:5386 read 0xe467d29 thru 0xe46c06f (0x4347 bytes) 2024-09-14T07:00:44.107 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.108 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.108 INFO:teuthology.orchestra.run.smithi007.stdout:5389 write 0x4022c15 thru 0x40271f9 (0x45e5 bytes) 2024-09-14T07:00:44.117 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.118 INFO:teuthology.orchestra.run.smithi007.stdout:5391 write 0x72ef37d thru 0x72fb9ea (0xc66e bytes) 2024-09-14T07:00:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:5392 read 0xaf55efc thru 0xaf63dc4 (0xdec9 bytes) 2024-09-14T07:00:44.128 INFO:teuthology.orchestra.run.smithi007.stdout:5394 write 0xc86699c thru 0xc868415 (0x1a7a bytes) 2024-09-14T07:00:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:5395 read 0xd688d57 thru 0xd68dd85 (0x502f bytes) 2024-09-14T07:00:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:5396 write 0xab06d90 thru 0xab1688a (0xfafb bytes) 2024-09-14T07:00:44.145 INFO:teuthology.orchestra.run.smithi007.stdout:5397 write 0x1ea6e43 thru 0x1eabc3c (0x4dfa bytes) 2024-09-14T07:00:44.152 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:44.151+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:5398 punch from 0x7a56e00 to 0x7a5dded, (0x6fed bytes) 2024-09-14T07:00:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:5399 read 0x744fbb0 thru 0x74507e5 (0xc36 bytes) 2024-09-14T07:00:44.155 INFO:teuthology.orchestra.run.smithi007.stdout:5401 write 0x2febde7 thru 0x2ff1115 (0x532f bytes) 2024-09-14T07:00:44.164 INFO:teuthology.orchestra.run.smithi007.stdout:5404 writesame 0xac9f296 thru 0xaca85b5 data_size 0x2c(0x9320 bytes) 2024-09-14T07:00:44.175 INFO:teuthology.orchestra.run.smithi007.stdout:5405 trunc from 0xeb7e747 to 0xd2dcad0 2024-09-14T07:00:44.182 INFO:teuthology.orchestra.run.smithi007.stdout:5406 trunc from 0xd2dcad0 to 0xdc91db8 2024-09-14T07:00:44.189 INFO:teuthology.orchestra.run.smithi007.stdout:5407 punch from 0x546e27d to 0x54721b4, (0x3f37 bytes) 2024-09-14T07:00:44.189 INFO:teuthology.orchestra.run.smithi007.stdout:5409 read 0xc07888b thru 0xc0816ba (0x8e30 bytes) 2024-09-14T07:00:44.189 INFO:teuthology.orchestra.run.smithi007.stdout:5410 write 0x8cdcb7a thru 0x8ceac05 (0xe08c bytes) 2024-09-14T07:00:44.199 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.199 INFO:teuthology.orchestra.run.smithi007.stdout:5413 writesame 0xdf3c7b thru 0xdf8f75 data_size 0x3(0x52fb bytes) 2024-09-14T07:00:44.209 INFO:teuthology.orchestra.run.smithi007.stdout:5415 punch from 0x8843311 to 0x8851775, (0xe464 bytes) 2024-09-14T07:00:44.211 INFO:teuthology.orchestra.run.smithi007.stdout:5416 trunc from 0xdc91db8 to 0xc1426ef 2024-09-14T07:00:44.217 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:44.217 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:44.217 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:44.222 INFO:teuthology.orchestra.run.smithi007.stdout:5417 write 0x25e8976 thru 0x25f282f (0x9eba bytes) 2024-09-14T07:00:44.225 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.225 INFO:teuthology.orchestra.run.smithi007.stdout:5419 writesame 0x3770992 thru 0x3777d09 data_size 0x14(0x7378 bytes) 2024-09-14T07:00:44.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:44.234 INFO:teuthology.orchestra.run.smithi007.stdout:5420 punch from 0x34f81dc to 0x34f9235, (0x1059 bytes) 2024-09-14T07:00:44.234 INFO:teuthology.orchestra.run.smithi007.stdout:5421 read 0x58efaf8 thru 0x58f1312 (0x181b bytes) 2024-09-14T07:00:44.234 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.234 INFO:teuthology.orchestra.run.smithi007.stdout:5423 writesame 0x399d7bc thru 0x399e73d data_size 0x5(0xf82 bytes) 2024-09-14T07:00:44.239 INFO:teuthology.orchestra.run.smithi007.stdout:5424 read 0x6178bfa thru 0x6187b73 (0xef7a bytes) 2024-09-14T07:00:44.250 INFO:teuthology.orchestra.run.smithi007.stdout:5425 trunc from 0xc1426ef to 0x255076a 2024-09-14T07:00:44.253 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:44.252+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:44.315 INFO:teuthology.orchestra.run.smithi007.stdout:5426 read 0x6b58c7 thru 0x6ba05f (0x4799 bytes) 2024-09-14T07:00:44.326 INFO:teuthology.orchestra.run.smithi007.stdout:5427 write 0x732d30d thru 0x732ff42 (0x2c36 bytes) 2024-09-14T07:00:44.336 INFO:teuthology.orchestra.run.smithi007.stdout:5428 read 0x4f3d763 thru 0x4f4b753 (0xdff1 bytes) 2024-09-14T07:00:44.336 INFO:teuthology.orchestra.run.smithi007.stdout:5429 write 0x68fc7a thru 0x695a21 (0x5da8 bytes) 2024-09-14T07:00:44.345 INFO:teuthology.orchestra.run.smithi007.stdout:5431 writesame 0x24dc561 thru 0x24dc5bd data_size 0x1(0x5d bytes) 2024-09-14T07:00:44.348 INFO:teuthology.orchestra.run.smithi007.stdout:5432 read 0x3a1c8bc thru 0x3a208d9 (0x401e bytes) 2024-09-14T07:00:44.348 INFO:teuthology.orchestra.run.smithi007.stdout:5433 write 0x19f878b thru 0x1a07e90 (0xf706 bytes) 2024-09-14T07:00:44.353 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:44.352+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:44.366 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:44.366 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:44.367 INFO:teuthology.orchestra.run.smithi007.stdout:5435 writesame 0xad3c6cc thru 0xad418e9 data_size 0x17(0x521e bytes) 2024-09-14T07:00:44.373 INFO:teuthology.orchestra.run.smithi007.stdout:5436 write 0xb0ff2d8 thru 0xb10ef9f (0xfcc8 bytes) 2024-09-14T07:00:44.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:44.384 INFO:teuthology.orchestra.run.smithi007.stdout:5437 writesame 0x772a1d3 thru 0x7738d14 data_size 0x1(0xeb42 bytes) 2024-09-14T07:00:44.403 INFO:teuthology.orchestra.run.smithi007.stdout:5439 read 0x7aab8b9 thru 0x7ab31e6 (0x792e bytes) 2024-09-14T07:00:44.403 INFO:teuthology.orchestra.run.smithi007.stdout:5440 punch from 0x3e18c66 to 0x3e25686, (0xca20 bytes) 2024-09-14T07:00:44.403 INFO:teuthology.orchestra.run.smithi007.stdout:5441 write 0x2f5d3d2 thru 0x2f61e16 (0x4a45 bytes) 2024-09-14T07:00:44.413 INFO:teuthology.orchestra.run.smithi007.stdout:5442 read 0x32e7e16 thru 0x32f28eb (0xaad6 bytes) 2024-09-14T07:00:44.413 INFO:teuthology.orchestra.run.smithi007.stdout:5446 read 0x1c8580 thru 0x1cecb5 (0x6736 bytes) 2024-09-14T07:00:44.414 INFO:teuthology.orchestra.run.smithi007.stdout:5448 read 0x9ca36a3 thru 0x9ca631c (0x2c7a bytes) 2024-09-14T07:00:44.414 INFO:teuthology.orchestra.run.smithi007.stdout:5449 punch from 0x23617e4 to 0x236e911, (0xd12d bytes) 2024-09-14T07:00:44.424 INFO:teuthology.orchestra.run.smithi007.stdout:5450 write 0xee5d782 thru 0xee655d5 (0x7e54 bytes) 2024-09-14T07:00:44.433 INFO:teuthology.orchestra.run.smithi007.stdout:5451 read 0x90aba3c thru 0x90b893a (0xceff bytes) 2024-09-14T07:00:44.449 INFO:teuthology.orchestra.run.smithi007.stdout:5452 trunc from 0xee655d6 to 0xaa24e8 2024-09-14T07:00:44.454 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:44.453+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:44.488 INFO:teuthology.orchestra.run.smithi007.stdout:5453 write 0xfaa509 thru 0xfb8cca (0xe7c2 bytes) 2024-09-14T07:00:44.505 INFO:teuthology.orchestra.run.smithi007.stdout:5454 write 0x2fd3f18 thru 0x2fd87c2 (0x48ab bytes) 2024-09-14T07:00:44.514 INFO:teuthology.orchestra.run.smithi007.stdout:5456 read 0x10f27c thru 0x11915f (0x9ee4 bytes) 2024-09-14T07:00:44.514 INFO:teuthology.orchestra.run.smithi007.stdout:5457 read 0xda7f18 thru 0xdae6b5 (0x679e bytes) 2024-09-14T07:00:44.524 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119746 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1298432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.977402382000001, 2024-09-14T07:00:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067798875999999994, 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119746 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545019+0000", 2024-09-14T07:00:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545020+0000", 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.782e-06 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545026+0000", 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9390000000000002e-06 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545031+0000", 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5770000000000004e-06 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612601+0000", 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067570682000000007 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612683+0000", 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1378999999999995e-05 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612817+0000", 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013415000000000001 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119747 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.977383963000001, 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068036012000000007, 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119747 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546792+0000", 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017554960000000001 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546797+0000", 2024-09-14T07:00:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1950000000000002e-06 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546808+0000", 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0675e-05 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546815+0000", 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612854+0000", 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066039158000000001 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612930+0000", 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.613073+0000", 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000142798 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119753 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1398784~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.975459801, 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067577770999999995, 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119753 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546961+0000", 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546993+0000", 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1860999999999999e-05 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547007+0000", 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4346e-05 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547016+0000", 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4709999999999996e-06 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547023+0000", 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4010000000000002e-06 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614322+0000", 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067299231000000001 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614391+0000", 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8826999999999997e-05 2024-09-14T07:00:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614538+0000", 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000147634 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119754 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1413120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.975369263000001, 2024-09-14T07:00:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067735654000000006, 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119754 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549973+0000", 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029215999999999999 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549978+0000", 2024-09-14T07:00:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3510000000000001e-06 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549988+0000", 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0037000000000001e-05 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549995+0000", 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9709999999999998e-06 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614569+0000", 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064573368000000006 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614650+0000", 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1909000000000005e-05 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614787+0000", 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013641799999999999 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119764 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1556480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.972197355999999, 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067672107999999995, 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119764 2024-09-14T07:00:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550224+0000", 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550226+0000", 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1720000000000001e-06 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550233+0000", 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9879999999999998e-06 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550238+0000", 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1819999999999997e-06 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617715+0000", 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067476628999999996 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617807+0000", 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2899999999999995e-05 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617895+0000", 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7837999999999995e-05 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119765 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.972174924000001, 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067829500000000001, 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119765 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7e-07 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550248+0000", 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1280000000000002e-06 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550256+0000", 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550261+0000", 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617926+0000", 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067665280999999994 2024-09-14T07:00:44.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617995+0000", 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9478000000000001e-05 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618075+0000", 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9666000000000006e-05 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119950 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 529408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.972152818000001, 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067992002999999995, 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119950 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550852+0000", 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058435400000000004 2024-09-14T07:00:44.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550857+0000", 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6979999999999997e-06 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550867+0000", 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0378e-05 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550874+0000", 2024-09-14T07:00:44.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8480000000000003e-06 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618106+0000", 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067231714999999997 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618178+0000", 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2572999999999998e-05 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618260+0000", 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1437e-05 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119970 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 816128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965497662000001, 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067745828999999994, 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119970 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556924+0000", 2024-09-14T07:00:44.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0410000000000001e-06 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556927+0000", 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9679999999999998e-06 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556930+0000", 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624487+0000", 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067557392999999993 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624551+0000", 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3861000000000004e-05 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624669+0000", 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117921 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119971 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965487722999999, 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067869065000000006, 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119971 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556934+0000", 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.096e-06 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556937+0000", 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.796e-06 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556939+0000", 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624708+0000", 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067768992 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624747+0000", 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8148999999999997e-05 2024-09-14T07:00:44.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624802+0000", 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.533e-05 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119972 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 844800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965478052000002, 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068010762000000002, 2024-09-14T07:00:44.565 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119972 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556943+0000", 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556944+0000", 2024-09-14T07:00:44.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556947+0000", 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9720000000000001e-06 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556949+0000", 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4899999999999999e-06 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624838+0000", 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067888899000000003 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624895+0000", 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7395000000000002e-05 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624953+0000", 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7745999999999999e-05 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.569 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119973 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 859136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965468170000001, 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068210898000000006, 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119973 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556953+0000", 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556954+0000", 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189999999999999e-06 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556957+0000", 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9490000000000001e-06 2024-09-14T07:00:44.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556959+0000", 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3870000000000002e-06 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624983+0000", 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068023726000000007 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625049+0000", 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6385999999999998e-05 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625163+0000", 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114221 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119974 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.574 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965458249000001, 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068338594000000003, 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119974 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556963+0000", 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556964+0000", 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556969+0000", 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.04e-06 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556974+0000", 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.639e-06 2024-09-14T07:00:44.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625194+0000", 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068220742000000001 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625246+0000", 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1844e-05 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625301+0000", 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4551e-05 2024-09-14T07:00:44.578 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119975 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965440009000002, 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068554198999999996, 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119975 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:44.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556983+0000", 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2560000000000001e-06 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556989+0000", 2024-09-14T07:00:44.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2210000000000002e-06 2024-09-14T07:00:44.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556994+0000", 2024-09-14T07:00:44.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7779999999999998e-06 2024-09-14T07:00:44.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625330+0000", 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068335911999999999 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625396+0000", 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6051999999999997e-05 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625535+0000", 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013860200000000001 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119976 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965419013999998, 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068656164000000006, 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119976 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557004+0000", 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557010+0000", 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1770000000000003e-06 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557016+0000", 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2650000000000004e-06 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625566+0000", 2024-09-14T07:00:44.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068550323999999996 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625604+0000", 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8398000000000001e-05 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625658+0000", 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3239000000000003e-05 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119977 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965338594999999, 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068713522999999999, 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119977 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.590 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:00:44.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557086+0000", 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3569999999999998e-06 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557094+0000", 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6829999999999997e-06 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557101+0000", 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625689+0000", 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068587882000000003 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625743+0000", 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4082000000000002e-05 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625795+0000", 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2657000000000001e-05 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119978 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 930816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965310002999999, 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068826410000000005, 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119978 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557113+0000", 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557120+0000", 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1079999999999998e-06 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557125+0000", 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625823+0000", 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068698928000000006 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625884+0000", 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0056999999999998e-05 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625937+0000", 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3383e-05 2024-09-14T07:00:44.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119979 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 945152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965288856000001, 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068940788000000003, 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119979 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:44.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5199999999999998e-07 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557134+0000", 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.018e-06 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557140+0000", 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557145+0000", 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7790000000000003e-06 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625966+0000", 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068820972999999994 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626019+0000", 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3828e-05 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626072+0000", 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3078000000000002e-05 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119980 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965268498, 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069053630000000005, 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119980 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557159+0000", 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557167+0000", 2024-09-14T07:00:44.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999996e-06 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557172+0000", 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626101+0000", 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068928552000000004 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626154+0000", 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3338000000000003e-05 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626206+0000", 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1657999999999999e-05 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119981 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.608 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965239848, 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069157629999999998, 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119981 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557183+0000", 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557190+0000", 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2249999999999997e-06 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557195+0000", 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3120000000000001e-06 2024-09-14T07:00:44.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626234+0000", 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069038926 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626286+0000", 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2604e-05 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626338+0000", 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1947000000000003e-05 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119982 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 988160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.965218695000001, 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069364004000000007, 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:44.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119982 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560989+0000", 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037867360000000002 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560994+0000", 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561004+0000", 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3680000000000008e-06 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:44.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561011+0000", 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1640000000000004e-06 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626366+0000", 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.065355286999999998 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626433+0000", 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6554e-05 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:44.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:44.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626566+0000", 2024-09-14T07:00:44.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132996 2024-09-14T07:00:44.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.618 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:44.618 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:44.618 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:44.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:00:44.620 INFO:teuthology.orchestra.run.smithi007.stdout:5459 writesame 0xe8ba69d thru 0xe8bfa02 data_size 0x46(0x5366 bytes) 2024-09-14T07:00:44.620 INFO:teuthology.orchestra.run.smithi007.stdout:5460 read 0x895a933 thru 0x8963267 (0x8935 bytes) 2024-09-14T07:00:44.620 INFO:teuthology.orchestra.run.smithi007.stdout:5461 read 0x230667a thru 0x23102b6 (0x9c3d bytes) 2024-09-14T07:00:44.620 INFO:teuthology.orchestra.run.smithi007.stdout:5462 write 0x2fd3037 thru 0x2fe00d7 (0xd0a1 bytes) 2024-09-14T07:00:44.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:5464 write 0x7867333 thru 0x786ed34 (0x7a02 bytes) 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:5466 write 0x12b7108 thru 0x12b9908 (0x2801 bytes) 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:5467 punch from 0x613b875 to 0x613eaff, (0x328a bytes) 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:5468 write 0x5e15deb thru 0x5e168eb (0xb01 bytes) 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:5469 write 0xebfcab1 thru 0xebfe2ae (0x17fe bytes) 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:5474 write 0x22c381c thru 0x22cb239 (0x7a1e bytes) 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:5475 writesame 0x777e816 thru 0x778d931 data_size 0x1a(0xf11c bytes) 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:5477 writesame 0x84cc430 thru 0x84d9dc2 data_size 0x7(0xd993 bytes) 2024-09-14T07:00:44.623 INFO:teuthology.orchestra.run.smithi007.stdout:5478 write 0x8e76477 thru 0x8e809a7 (0xa531 bytes) 2024-09-14T07:00:44.624 INFO:teuthology.orchestra.run.smithi007.stdout:5479 read 0x86310ef thru 0x8637cc1 (0x6bd3 bytes) 2024-09-14T07:00:44.624 INFO:teuthology.orchestra.run.smithi007.stdout:5480 punch from 0x199937d to 0x19995be, (0x241 bytes) 2024-09-14T07:00:44.624 INFO:teuthology.orchestra.run.smithi007.stdout:5481 write 0x840c22a thru 0x8410492 (0x4269 bytes) 2024-09-14T07:00:44.624 INFO:teuthology.orchestra.run.smithi007.stdout:5483 read 0x2f707d1 thru 0x2f74162 (0x3992 bytes) 2024-09-14T07:00:44.624 INFO:teuthology.orchestra.run.smithi007.stdout:5484 writesame 0xcf95bfd thru 0xcfa3ec1 data_size 0x25(0xe2c5 bytes) 2024-09-14T07:00:44.624 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:44.618+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:44.624 INFO:teuthology.orchestra.run.smithi007.stdout:5485 write 0x9d965f0 thru 0x9d9d8c9 (0x72da bytes) 2024-09-14T07:00:44.628 INFO:teuthology.orchestra.run.smithi007.stdout:5486 write 0x22f1c3 thru 0x233427 (0x4265 bytes) 2024-09-14T07:00:44.638 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.638 INFO:teuthology.orchestra.run.smithi007.stdout:5491 read 0x5e40d81 thru 0x5e4aa15 (0x9c95 bytes) 2024-09-14T07:00:44.639 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.639 INFO:teuthology.orchestra.run.smithi007.stdout:5493 write 0x8918126 thru 0x8922631 (0xa50c bytes) 2024-09-14T07:00:44.650 INFO:teuthology.orchestra.run.smithi007.stdout:5494 writesame 0x2ccf318 thru 0x2cd79a8 data_size 0x1(0x8691 bytes) 2024-09-14T07:00:44.667 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.672 INFO:teuthology.orchestra.run.smithi007.stdout:5496 trunc from 0xebfe2af to 0xb73026c 2024-09-14T07:00:44.685 INFO:teuthology.orchestra.run.smithi007.stdout:5498 read 0x2f6c8c7 thru 0x2f741bc (0x78f6 bytes) 2024-09-14T07:00:44.696 INFO:teuthology.orchestra.run.smithi007.stdout:5499 trunc from 0xb73026c to 0x25d510f 2024-09-14T07:00:44.720 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:44.719+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:44.721 INFO:teuthology.orchestra.run.smithi007.stdout:5500 read 0x1b94f73 thru 0x1b98e97 (0x3f25 bytes) 2024-09-14T07:00:44.721 INFO:teuthology.orchestra.run.smithi007.stdout:5502 read 0xb165c6 thru 0xb16a53 (0x48e bytes) 2024-09-14T07:00:44.721 INFO:teuthology.orchestra.run.smithi007.stdout:5503 read 0x1b85a9c thru 0x1b91404 (0xb969 bytes) 2024-09-14T07:00:44.737 INFO:teuthology.orchestra.run.smithi007.stdout:5504 write 0xb07adbd thru 0xb07d5dc (0x2820 bytes) 2024-09-14T07:00:44.748 INFO:teuthology.orchestra.run.smithi007.stdout:5505 punch from 0x7cc54a4 to 0x7cc9e5e, (0x49ba bytes) 2024-09-14T07:00:44.748 INFO:teuthology.orchestra.run.smithi007.stdout:5506 read 0x8397dd6 thru 0x839e246 (0x6471 bytes) 2024-09-14T07:00:44.748 INFO:teuthology.orchestra.run.smithi007.stdout:5507 read 0x9d31cbf thru 0x9d3d96b (0xbcad bytes) 2024-09-14T07:00:44.755 INFO:teuthology.orchestra.run.smithi007.stdout:5508 trunc from 0xb07d5dd to 0x4bf6412 2024-09-14T07:00:44.772 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:44.772 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:44.772 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:44.778 INFO:teuthology.orchestra.run.smithi007.stdout:5509 writesame 0x84f5d0a thru 0x85013c4 data_size 0x1f(0xb6bb bytes) 2024-09-14T07:00:44.784 INFO:teuthology.orchestra.run.smithi007.stdout:5510 trunc from 0x85013c5 to 0x3bbbf99 2024-09-14T07:00:44.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:44.790 INFO:teuthology.orchestra.run.smithi007.stdout:5512 writesame 0x3ad0da0 thru 0x3adbe9d data_size 0x73(0xb0fe bytes) 2024-09-14T07:00:44.801 INFO:teuthology.orchestra.run.smithi007.stdout:5513 punch from 0x2a673df to 0x2a70263, (0x8e84 bytes) 2024-09-14T07:00:44.801 INFO:teuthology.orchestra.run.smithi007.stdout:5514 read 0x3686ce4 thru 0x368a037 (0x3354 bytes) 2024-09-14T07:00:44.811 INFO:teuthology.orchestra.run.smithi007.stdout:5515 write 0x776cc17 thru 0x777781a (0xac04 bytes) 2024-09-14T07:00:44.821 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:44.820+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:44.834 INFO:teuthology.orchestra.run.smithi007.stdout:5516 write 0xd73a451 thru 0xd749549 (0xf0f9 bytes) 2024-09-14T07:00:44.845 INFO:teuthology.orchestra.run.smithi007.stdout:5517 read 0x25005ff thru 0x250a9b6 (0xa3b8 bytes) 2024-09-14T07:00:44.845 INFO:teuthology.orchestra.run.smithi007.stdout:5518 write 0x4ba0c39 thru 0x4bac1dd (0xb5a5 bytes) 2024-09-14T07:00:44.868 INFO:teuthology.orchestra.run.smithi007.stdout:5519 trunc from 0xd74954a to 0x29ba433 2024-09-14T07:00:44.895 INFO:teuthology.orchestra.run.smithi007.stdout:5521 writesame 0x5bc11eb thru 0x5bc8ec6 data_size 0x7a(0x7cdc bytes) 2024-09-14T07:00:44.904 INFO:teuthology.orchestra.run.smithi007.stdout:5522 trunc from 0x5bc8ec7 to 0xcb00518 2024-09-14T07:00:44.909 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:44.917 INFO:teuthology.orchestra.run.smithi007.stdout:5525 write 0xe5ef9f3 thru 0xe5f0861 (0xe6f bytes) 2024-09-14T07:00:44.921 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:44.921+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:44.928 INFO:teuthology.orchestra.run.smithi007.stdout:5526 write 0xe6493bd thru 0xe64fde7 (0x6a2b bytes) 2024-09-14T07:00:44.936 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:44.936 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:44.941 INFO:teuthology.orchestra.run.smithi007.stdout:5527 trunc from 0xe64fde8 to 0x8cbad53 2024-09-14T07:00:44.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:44.954 INFO:teuthology.orchestra.run.smithi007.stdout:5528 write 0x8a86b4e thru 0x8a8fe9b (0x934e bytes) 2024-09-14T07:00:44.965 INFO:teuthology.orchestra.run.smithi007.stdout:5530 trunc from 0x8cbad53 to 0x9fab49c 2024-09-14T07:00:44.972 INFO:teuthology.orchestra.run.smithi007.stdout:5531 read 0x5bd538f thru 0x5bd7401 (0x2073 bytes) 2024-09-14T07:00:44.973 INFO:teuthology.orchestra.run.smithi007.stdout:5532 writesame 0x9a723fc thru 0x9a7a2c2 data_size 0x1(0x7ec7 bytes) 2024-09-14T07:00:44.987 INFO:teuthology.orchestra.run.smithi007.stdout:5533 read 0x5fcc373 thru 0x5fd6d1f (0xa9ad bytes) 2024-09-14T07:00:44.987 INFO:teuthology.orchestra.run.smithi007.stdout:5534 write 0x1050b65 thru 0x105c6b2 (0xbb4e bytes) 2024-09-14T07:00:44.990 INFO:teuthology.orchestra.run.smithi007.stdout:5535 read 0x5ed771e thru 0x5ee4959 (0xd23c bytes) 2024-09-14T07:00:44.990 INFO:teuthology.orchestra.run.smithi007.stdout:5536 punch from 0x9369be to 0x93afa0, (0x45e2 bytes) 2024-09-14T07:00:44.990 INFO:teuthology.orchestra.run.smithi007.stdout:5537 read 0x66e359b thru 0x66f034d (0xcdb3 bytes) 2024-09-14T07:00:44.990 INFO:teuthology.orchestra.run.smithi007.stdout:5538 read 0x64c19c1 thru 0x64c7267 (0x58a7 bytes) 2024-09-14T07:00:44.990 INFO:teuthology.orchestra.run.smithi007.stdout:5540 read 0x1cc17c3 thru 0x1cc82c5 (0x6b03 bytes) 2024-09-14T07:00:44.999 INFO:teuthology.orchestra.run.smithi007.stdout:5541 writesame 0xd33818b thru 0xd3407a9 data_size 0x69(0x861f bytes) 2024-09-14T07:00:44.999 INFO:teuthology.orchestra.run.smithi007.stdout:5542 read 0xc8fd143 thru 0xc905f29 (0x8de7 bytes) 2024-09-14T07:00:45.000 INFO:teuthology.orchestra.run.smithi007.stdout:5543 write 0xadeb2b2 thru 0xadf0ddb (0x5b2a bytes) 2024-09-14T07:00:45.016 INFO:teuthology.orchestra.run.smithi007.stdout:5544 trunc from 0xd3407aa to 0x6c36541 2024-09-14T07:00:45.022 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:45.022+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:45.036 INFO:teuthology.orchestra.run.smithi007.stdout:5546 write 0x6dd9fa5 thru 0x6ddb1ca (0x1226 bytes) 2024-09-14T07:00:45.054 INFO:teuthology.orchestra.run.smithi007.stdout:5549 write 0xbd943cf thru 0xbd9d381 (0x8fb3 bytes) 2024-09-14T07:00:45.064 INFO:teuthology.orchestra.run.smithi007.stdout:5550 punch from 0x85ad52b to 0x85adc38, (0x70d bytes) 2024-09-14T07:00:45.065 INFO:teuthology.orchestra.run.smithi007.stdout:5551 writesame 0x99e3dd5 thru 0x99ec9ae data_size 0x6(0x8bda bytes) 2024-09-14T07:00:45.074 INFO:teuthology.orchestra.run.smithi007.stdout:5553 writesame 0x6fa90bf thru 0x6faa314 data_size 0x1(0x1256 bytes) 2024-09-14T07:00:45.080 INFO:teuthology.orchestra.run.smithi007.stdout:5554 write 0xa729089 thru 0xa73807a (0xeff2 bytes) 2024-09-14T07:00:45.099 INFO:teuthology.orchestra.run.smithi007.stdout:5555 write 0xe814061 thru 0xe8228f7 (0xe897 bytes) 2024-09-14T07:00:45.114 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:45.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.358986978000004, 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:00:45.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:00:45.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 65.760574386000002, 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:00:45.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:00:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:45.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.128 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 65.754914743, 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:00:45.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:00:45.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:00:45.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.449078464999999, 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:45.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.139 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.449047673000003, 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.449024303000002, 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:00:45.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.448217794000001, 2024-09-14T07:00:45.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:00:45.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:00:45.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.448195401, 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.153 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:00:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:00:45.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:00:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:00:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.157 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.448175261000003, 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:00:45.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.448025106000003, 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.166 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.447969552000004, 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:00:45.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.447936511000002, 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:00:45.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:45.175 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.445583425999999, 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:45.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:00:45.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.445560860999997, 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.180 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:45.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:00:45.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.444703865999998, 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:45.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.188 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.443486845000002, 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:45.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:00:45.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.443359004999998, 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.193 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:00:45.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:00:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:00:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:00:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:00:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.443338912000002, 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.201 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:00:45.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.442312018999999, 2024-09-14T07:00:45.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:00:45.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138497 3.14 3:298b17d3:::rbd_data.114f74bd3bbf.00000000000002d7:head [set-alloc-hint object_size 262144 write_size 262144,write 218355~43789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.794804737, 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97686345699999999, 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:45.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138497 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317983+0000", 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318003+0000", 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0610000000000001e-05 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318022+0000", 2024-09-14T07:00:45.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8627e-05 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318030+0000", 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318101+0000", 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1162999999999996e-05 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318108+0000", 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291257+0000", 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97314914100000005 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291396+0000", 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013883799999999999 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291573+0000", 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017716800000000001 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.293147+0000", 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015740400000000001 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294753+0000", 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00160597 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294792+0000", 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8758999999999999e-05 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294844+0000", 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2188000000000002e-05 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:45.216 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:45.217 INFO:teuthology.orchestra.run.smithi007.stdout:5556 trunc from 0xe8228f8 to 0x4fe3e2b 2024-09-14T07:00:45.218 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:5557 read 0x2f3142d thru 0x2f4050a (0xf0de bytes) 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:5558 punch from 0x4635252 to 0x463b1d9, (0x5f87 bytes) 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:5562 write 0x4c70934 thru 0x4c7497d (0x404a bytes) 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:5563 write 0x5e94a5d thru 0x5e96df1 (0x2395 bytes) 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:5564 trunc from 0x5e96df2 to 0xaa4105f 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:5566 trunc from 0xaa4105f to 0x12edf34 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:5568 read 0xecb77b thru 0xed3945 (0x81cb bytes) 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:5570 trunc from 0x12edf34 to 0x7dc53fa 2024-09-14T07:00:45.220 INFO:teuthology.orchestra.run.smithi007.stdout:5571 read 0x47dd32 thru 0x486ae3 (0x8db2 bytes) 2024-09-14T07:00:45.221 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:45.217+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:45.224 INFO:teuthology.orchestra.run.smithi007.stdout:5572 write 0xd435bcb thru 0xd4420cb (0xc501 bytes) 2024-09-14T07:00:45.233 INFO:teuthology.orchestra.run.smithi007.stdout:5573 write 0x741c18b thru 0x7422f65 (0x6ddb bytes) 2024-09-14T07:00:45.241 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.241 INFO:teuthology.orchestra.run.smithi007.stdout:5575 punch from 0x6523fc5 to 0x652edcd, (0xae08 bytes) 2024-09-14T07:00:45.241 INFO:teuthology.orchestra.run.smithi007.stdout:5576 write 0x2b3429b thru 0x2b42a39 (0xe79f bytes) 2024-09-14T07:00:45.251 INFO:teuthology.orchestra.run.smithi007.stdout:5577 read 0xb473cd4 thru 0xb47647a (0x27a7 bytes) 2024-09-14T07:00:45.251 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.251 INFO:teuthology.orchestra.run.smithi007.stdout:5581 write 0xacac8af thru 0xacb9b52 (0xd2a4 bytes) 2024-09-14T07:00:45.259 INFO:teuthology.orchestra.run.smithi007.stdout:5582 write 0x9fe2b11 thru 0x9fe5ca1 (0x3191 bytes) 2024-09-14T07:00:45.269 INFO:teuthology.orchestra.run.smithi007.stdout:5584 trunc from 0xd4420cc to 0xc1a62d2 2024-09-14T07:00:45.280 INFO:teuthology.orchestra.run.smithi007.stdout:5585 write 0x118e548 thru 0x1194b2b (0x65e4 bytes) 2024-09-14T07:00:45.289 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.289 INFO:teuthology.orchestra.run.smithi007.stdout:5588 punch from 0x1e748c4 to 0x1e7f493, (0xabcf bytes) 2024-09-14T07:00:45.289 INFO:teuthology.orchestra.run.smithi007.stdout:5589 write 0xb25ebc6 thru 0xb262583 (0x39be bytes) 2024-09-14T07:00:45.305 INFO:teuthology.orchestra.run.smithi007.stdout:5590 writesame 0xd9fffe7 thru 0xda02db7 data_size 0x1(0x2dd1 bytes) 2024-09-14T07:00:45.305 INFO:teuthology.orchestra.run.smithi007.stdout:5591 read 0x7876d87 thru 0x7879c08 (0x2e82 bytes) 2024-09-14T07:00:45.310 INFO:teuthology.orchestra.run.smithi007.stdout:5592 trunc from 0xda02db8 to 0x90e3021 2024-09-14T07:00:45.318 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:45.317+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:45.333 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:45.333 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:45.333 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:45.335 INFO:teuthology.orchestra.run.smithi007.stdout:5594 write 0xd9f163b thru 0xd9fe0d8 (0xca9e bytes) 2024-09-14T07:00:45.342 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:45.344 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.344 INFO:teuthology.orchestra.run.smithi007.stdout:5597 punch from 0x9c70367 to 0x9c7da18, (0xd6b1 bytes) 2024-09-14T07:00:45.344 INFO:teuthology.orchestra.run.smithi007.stdout:5598 write 0x2bc63a8 thru 0x2bd20b8 (0xbd11 bytes) 2024-09-14T07:00:45.353 INFO:teuthology.orchestra.run.smithi007.stdout:5599 read 0x2af4cb8 thru 0x2afe15d (0x94a6 bytes) 2024-09-14T07:00:45.354 INFO:teuthology.orchestra.run.smithi007.stdout:5601 punch from 0xc89de13 to 0xc8a6ec5, (0x90b2 bytes) 2024-09-14T07:00:45.354 INFO:teuthology.orchestra.run.smithi007.stdout:5602 read 0xceae410 thru 0xceae60d (0x1fe bytes) 2024-09-14T07:00:45.354 INFO:teuthology.orchestra.run.smithi007.stdout:5603 write 0xca4139a thru 0xca49bc3 (0x882a bytes) 2024-09-14T07:00:45.362 INFO:teuthology.orchestra.run.smithi007.stdout:5604 read 0x8071512 thru 0x807e8ce (0xd3bd bytes) 2024-09-14T07:00:45.362 INFO:teuthology.orchestra.run.smithi007.stdout:5605 read 0x1204b7a thru 0x120692c (0x1db3 bytes) 2024-09-14T07:00:45.362 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.362 INFO:teuthology.orchestra.run.smithi007.stdout:5608 read 0x1ce462b thru 0x1cecaed (0x84c3 bytes) 2024-09-14T07:00:45.375 INFO:teuthology.orchestra.run.smithi007.stdout:5610 trunc from 0xd9fe0d9 to 0x1e8ba3c 2024-09-14T07:00:45.407 INFO:teuthology.orchestra.run.smithi007.stdout:5611 write 0x97cddd9 thru 0x97cf2d8 (0x1500 bytes) 2024-09-14T07:00:45.413 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.413 INFO:teuthology.orchestra.run.smithi007.stdout:5614 write 0x5074917 thru 0x508217a (0xd864 bytes) 2024-09-14T07:00:45.422 INFO:teuthology.orchestra.run.smithi007.stdout:5615 punch from 0x4ad11f0 to 0x4ad8aa9, (0x78b9 bytes) 2024-09-14T07:00:45.422 INFO:teuthology.orchestra.run.smithi007.stdout:5616 read 0x4d6155f thru 0x4d70ff2 (0xfa94 bytes) 2024-09-14T07:00:45.422 INFO:teuthology.orchestra.run.smithi007.stdout:5617 punch from 0x1678808 to 0x167d8fa, (0x50f2 bytes) 2024-09-14T07:00:45.425 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:45.425+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:45.433 INFO:teuthology.orchestra.run.smithi007.stdout:5618 write 0xe303ae7 thru 0xe30fb8c (0xc0a6 bytes) 2024-09-14T07:00:45.443 INFO:teuthology.orchestra.run.smithi007.stdout:5620 write 0x549975a thru 0x54a5645 (0xbeec bytes) 2024-09-14T07:00:45.452 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.459 INFO:teuthology.orchestra.run.smithi007.stdout:5622 trunc from 0xe30fb8d to 0x87afb09 2024-09-14T07:00:45.471 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.482 INFO:teuthology.orchestra.run.smithi007.stdout:5624 write 0xdb23375 thru 0xdb2950c (0x6198 bytes) 2024-09-14T07:00:45.490 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:45.491 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:45.491 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:45.491 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:45.491 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:45.491 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.491 INFO:teuthology.orchestra.run.smithi007.stdout:5626 write 0x9f47d28 thru 0x9f5161a (0x98f3 bytes) 2024-09-14T07:00:45.500 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:45.506 INFO:teuthology.orchestra.run.smithi007.stdout:5628 writesame 0xea74495 thru 0xea78ab8 data_size 0x43(0x4624 bytes) 2024-09-14T07:00:45.507 INFO:teuthology.orchestra.run.smithi007.stdout:5629 read 0xa1b170 thru 0xa1c90a (0x179b bytes) 2024-09-14T07:00:45.507 INFO:teuthology.orchestra.run.smithi007.stdout:5630 read 0x2691f17 thru 0x2698fda (0x70c4 bytes) 2024-09-14T07:00:45.507 INFO:teuthology.orchestra.run.smithi007.stdout:5632 write 0xd9f5f04 thru 0xd9f9820 (0x391d bytes) 2024-09-14T07:00:45.515 INFO:teuthology.orchestra.run.smithi007.stdout:5633 read 0x7c2ae4c thru 0x7c2c876 (0x1a2b bytes) 2024-09-14T07:00:45.520 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:45.519+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:45.531 INFO:teuthology.orchestra.run.smithi007.stdout:5634 trunc from 0xea78ab9 to 0x91a057 2024-09-14T07:00:45.551 INFO:teuthology.orchestra.run.smithi007.stdout:5635 read 0x918ed8 thru 0x91a056 (0x117f bytes) 2024-09-14T07:00:45.556 INFO:teuthology.orchestra.run.smithi007.stdout:5638 trunc from 0x91a057 to 0x4fa0af8 2024-09-14T07:00:45.566 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T07:00:45.575 INFO:teuthology.orchestra.run.smithi007.stdout:5640 write 0x78874e7 thru 0x788c5e1 (0x50fb bytes) 2024-09-14T07:00:45.585 INFO:teuthology.orchestra.run.smithi007.stdout:5641 read 0x3c3d216 thru 0x3c471f4 (0x9fdf bytes) 2024-09-14T07:00:45.586 INFO:teuthology.orchestra.run.smithi007.stdout:5643 writesame 0x8494f3 thru 0x857f27 data_size 0x1(0xea35 bytes) 2024-09-14T07:00:45.610 INFO:teuthology.orchestra.run.smithi007.stdout:5644 write 0xc0a90ce thru 0xc0ab5a1 (0x24d4 bytes) 2024-09-14T07:00:45.618 INFO:teuthology.orchestra.run.smithi007.stdout:5645 writesame 0xa0b7d0 thru 0xa0d397 data_size 0x1c(0x1bc8 bytes) 2024-09-14T07:00:45.621 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:45.620+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:45.625 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.626 INFO:teuthology.orchestra.run.smithi007.stdout:5647 write 0x62d6caa thru 0x62e68c6 (0xfc1d bytes) 2024-09-14T07:00:45.644 INFO:teuthology.orchestra.run.smithi007.stdout:5648 trunc from 0xc0ab5a2 to 0x2bd4500 2024-09-14T07:00:45.655 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:45.655 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:45.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:45.655 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:45.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.656 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131358 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.656 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:45.656 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155334319000001, 2024-09-14T07:00:45.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030805583, 2024-09-14T07:00:45.656 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.656 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.656 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.656 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131358 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498000+0000", 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:00:45.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498004+0000", 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2569999999999999e-06 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498007+0000", 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.507e-06 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499958+0000", 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001950989 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499962+0000", 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528792+0000", 2024-09-14T07:00:45.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288306389999999 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528804+0000", 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2096000000000001e-05 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131359 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155314909000001, 2024-09-14T07:00:45.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308363920000001, 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131359 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:45.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:45.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498019+0000", 2024-09-14T07:00:45.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:00:45.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498020+0000", 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4529999999999999e-06 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498023+0000", 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3450000000000002e-06 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498026+0000", 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.661e-06 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499995+0000", 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00196926 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500002+0000", 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6930000000000001e-06 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528842+0000", 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288401300000001 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528855+0000", 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2595999999999999e-05 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.667 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131360 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155303906, 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030841943, 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131360 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498030+0000", 2024-09-14T07:00:45.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498031+0000", 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1119999999999999e-06 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498034+0000", 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498036+0000", 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.599e-06 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500030+0000", 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001993613 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500034+0000", 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1060000000000002e-06 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528867+0000", 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288327429999999 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528871+0000", 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.673e-06 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.672 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131361 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155283161, 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308644520000001, 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131361 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:45.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498052+0000", 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.24e-06 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498055+0000", 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1609999999999999e-06 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498058+0000", 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.05e-06 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500066+0000", 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020081169999999998 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500075+0000", 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0559999999999994e-06 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528903+0000", 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288276810000001 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528915+0000", 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1938e-05 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.677 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131362 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155264379999998, 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030860661, 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131362 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498075+0000", 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5639999999999996e-06 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498079+0000", 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498081+0000", 2024-09-14T07:00:45.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8660000000000002e-06 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500101+0000", 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020191219999999999 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500104+0000", 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6890000000000002e-06 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528925+0000", 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028820971 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528930+0000", 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3390000000000003e-06 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131363 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:45.682 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155248132000001, 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030874107, 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131363 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498086+0000", 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1009999999999999e-06 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498090+0000", 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5609999999999999e-06 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498093+0000", 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7360000000000001e-06 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500136+0000", 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020436030000000002 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500145+0000", 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9530000000000005e-06 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528948+0000", 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028803103 2024-09-14T07:00:45.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528959+0000", 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0808999999999999e-05 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131364 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155231750999999, 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308734129999999, 2024-09-14T07:00:45.687 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131364 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498102+0000", 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498103+0000", 2024-09-14T07:00:45.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498107+0000", 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4089999999999999e-06 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498110+0000", 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7760000000000002e-06 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500177+0000", 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020678200000000002 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500183+0000", 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4979999999999997e-06 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528970+0000", 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287867049999999 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528975+0000", 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1959999999999999e-06 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131365 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155209259999999, 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886024, 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.692 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131365 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498125+0000", 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.17e-06 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498129+0000", 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498131+0000", 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.666e-06 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500210+0000", 2024-09-14T07:00:45.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020789559999999999 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500215+0000", 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528993+0000", 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028777174 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529010+0000", 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7419999999999999e-05 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131366 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155190961999999, 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308831409999999, 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.697 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131366 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498143+0000", 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498144+0000", 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.474e-06 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498147+0000", 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.371e-06 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498150+0000", 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6410000000000002e-06 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500239+0000", 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002088842 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500243+0000", 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6689999999999999e-06 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529020+0000", 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287776420000001 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529025+0000", 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2440000000000001e-06 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131367 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155179902, 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030895135, 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.702 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131367 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498154+0000", 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2100000000000001e-07 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498155+0000", 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1310000000000001e-06 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498157+0000", 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7870000000000002e-06 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498160+0000", 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9009999999999998e-06 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500264+0000", 2024-09-14T07:00:45.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021031489999999999 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500268+0000", 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0929999999999996e-06 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529043+0000", 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287752139999999 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529048+0000", 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6389999999999997e-06 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131368 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.15515577, 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308865389999999, 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.707 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131368 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498178+0000", 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8200000000000001e-07 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498179+0000", 2024-09-14T07:00:45.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6929999999999999e-06 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498183+0000", 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9439999999999998e-06 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498188+0000", 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9409999999999998e-06 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500287+0000", 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020982269999999998 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500290+0000", 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6349999999999999e-06 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529059+0000", 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028768648 2024-09-14T07:00:45.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529064+0000", 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.169e-06 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131369 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155140578000001, 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886017, 2024-09-14T07:00:45.712 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131369 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.28e-07 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498194+0000", 2024-09-14T07:00:45.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1459999999999999e-06 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498197+0000", 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0620000000000001e-06 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498200+0000", 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6000000000000001e-06 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500310+0000", 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021101840000000002 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500314+0000", 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.126e-06 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529074+0000", 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.02875961 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529079+0000", 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0610000000000003e-06 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131370 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155124785000002, 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308930119999999, 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.717 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131370 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498209+0000", 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498210+0000", 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.187e-06 2024-09-14T07:00:45.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498213+0000", 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.151e-06 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498216+0000", 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8289999999999999e-06 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500340+0000", 2024-09-14T07:00:45.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021244229999999998 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500349+0000", 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529096+0000", 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287473309999999 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.721 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529101+0000", 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131371 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155106405000002, 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308901880000001, 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.722 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131371 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:45.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498229+0000", 2024-09-14T07:00:45.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1110000000000002e-06 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498234+0000", 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498239+0000", 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.651e-06 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.725 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500382+0000", 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021424780000000002 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500389+0000", 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529112+0000", 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287229410000001 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529117+0000", 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9520000000000002e-06 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131372 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155085288999999, 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030884366, 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.727 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131372 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8599999999999999e-07 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498251+0000", 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498257+0000", 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6370000000000002e-06 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498262+0000", 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500418+0000", 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021561800000000002 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500423+0000", 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1520000000000002e-06 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529127+0000", 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287049150000001 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529132+0000", 2024-09-14T07:00:45.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.865e-06 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131373 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155052225999999, 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308673479999999, 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.732 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131373 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:45.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498283+0000", 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498290+0000", 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498295+0000", 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3759999999999999e-06 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500441+0000", 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021462849999999999 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.735 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500445+0000", 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8310000000000003e-06 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529142+0000", 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028697363 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529148+0000", 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:45.736 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131374 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.155018609999999, 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308477090000001, 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.737 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131374 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498322+0000", 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5760000000000002e-06 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498328+0000", 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4609999999999996e-06 2024-09-14T07:00:45.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498334+0000", 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7130000000000002e-06 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500464+0000", 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002130617 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500468+0000", 2024-09-14T07:00:45.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7340000000000002e-06 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529157+0000", 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286893669999999 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529162+0000", 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.741 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131375 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.154978473, 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308296320000001, 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131375 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.742 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:45.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498358+0000", 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.317e-06 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498364+0000", 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5930000000000002e-06 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498369+0000", 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3630000000000002e-06 2024-09-14T07:00:45.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500488+0000", 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021187699999999999 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500492+0000", 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529179+0000", 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286873590000001 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529184+0000", 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131376 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.154950933999999, 2024-09-14T07:00:45.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030817713, 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131376 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:45.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498383+0000", 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.748 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498385+0000", 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.091e-06 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498391+0000", 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1070000000000001e-06 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498396+0000", 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4670000000000003e-06 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.749 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500514+0000", 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021177449999999999 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500518+0000", 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8770000000000003e-06 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529195+0000", 2024-09-14T07:00:45.750 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028676938 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529200+0000", 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131377 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 29.154911409, 2024-09-14T07:00:45.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308008749999999, 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131377 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498425+0000", 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5620000000000002e-06 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498431+0000", 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2539999999999998e-06 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498436+0000", 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.31e-06 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500539+0000", 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021025459999999998 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500543+0000", 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.597e-06 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529218+0000", 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286753930000001 2024-09-14T07:00:45.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529223+0000", 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.758e-06 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:45.756 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:45.759 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:45.759 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:45.757+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:45.760 INFO:teuthology.orchestra.run.smithi007.stdout:5649 read 0x902ce thru 0x9f2e1 (0xf014 bytes) 2024-09-14T07:00:45.760 INFO:teuthology.orchestra.run.smithi007.stdout:5651 trunc from 0x2bd4500 to 0x652102a 2024-09-14T07:00:45.760 INFO:teuthology.orchestra.run.smithi007.stdout:5655 write 0xa7c8c1d thru 0xa7d1c32 (0x9016 bytes) 2024-09-14T07:00:45.760 INFO:teuthology.orchestra.run.smithi007.stdout:5656 read 0x1a22233 thru 0x1a2eb06 (0xc8d4 bytes) 2024-09-14T07:00:45.760 INFO:teuthology.orchestra.run.smithi007.stdout:5657 read 0xecb47a thru 0xed72fb (0xbe82 bytes) 2024-09-14T07:00:45.760 INFO:teuthology.orchestra.run.smithi007.stdout:5658 write 0x8375945 thru 0x837b926 (0x5fe2 bytes) 2024-09-14T07:00:45.760 INFO:teuthology.orchestra.run.smithi007.stdout:5663 read 0x9c166c7 thru 0x9c19809 (0x3143 bytes) 2024-09-14T07:00:45.761 INFO:teuthology.orchestra.run.smithi007.stdout:5664 write 0x414d10b thru 0x415a451 (0xd347 bytes) 2024-09-14T07:00:45.761 INFO:teuthology.orchestra.run.smithi007.stdout:5665 write 0x555e9f9 thru 0x556e728 (0xfd30 bytes) 2024-09-14T07:00:45.761 INFO:teuthology.orchestra.run.smithi007.stdout:5666 write 0xbdc614a thru 0xbdcbffb (0x5eb2 bytes) 2024-09-14T07:00:45.761 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.761 INFO:teuthology.orchestra.run.smithi007.stdout:5670 write 0x8fe4bae thru 0x8fe60e5 (0x1538 bytes) 2024-09-14T07:00:45.761 INFO:teuthology.orchestra.run.smithi007.stdout:5671 writesame 0xb383390 thru 0xb38fc73 data_size 0x5c(0xc8e4 bytes) 2024-09-14T07:00:45.761 INFO:teuthology.orchestra.run.smithi007.stdout:5674 read 0x80887a1 thru 0x8089ce2 (0x1542 bytes) 2024-09-14T07:00:45.761 INFO:teuthology.orchestra.run.smithi007.stdout:5676 trunc from 0xbdcbffc to 0xdc60018 2024-09-14T07:00:45.761 INFO:teuthology.orchestra.run.smithi007.stdout:5677 write 0x705015c thru 0x70554af (0x5354 bytes) 2024-09-14T07:00:45.765 INFO:teuthology.orchestra.run.smithi007.stdout:5678 write 0x1a4485f thru 0x1a4500d (0x7af bytes) 2024-09-14T07:00:45.769 INFO:teuthology.orchestra.run.smithi007.stdout:5679 writesame 0x7bdb78c thru 0x7be35b8 data_size 0x61(0x7e2d bytes) 2024-09-14T07:00:45.778 INFO:teuthology.orchestra.run.smithi007.stdout:5680 read 0xd6d4741 thru 0xd6dede2 (0xa6a2 bytes) 2024-09-14T07:00:45.778 INFO:teuthology.orchestra.run.smithi007.stdout:5681 writesame 0x2c53366 thru 0x2c5f034 data_size 0x7(0xbccf bytes) 2024-09-14T07:00:45.789 INFO:teuthology.orchestra.run.smithi007.stdout:5683 read 0x80a86ea thru 0x80aee0f (0x6726 bytes) 2024-09-14T07:00:45.789 INFO:teuthology.orchestra.run.smithi007.stdout:5684 read 0x485543e thru 0x48608f0 (0xb4b3 bytes) 2024-09-14T07:00:45.789 INFO:teuthology.orchestra.run.smithi007.stdout:5685 write 0xbfd4f75 thru 0xbfd9a3a (0x4ac6 bytes) 2024-09-14T07:00:45.805 INFO:teuthology.orchestra.run.smithi007.stdout:5686 trunc from 0xdc60018 to 0x6f6ae92 2024-09-14T07:00:45.826 INFO:teuthology.orchestra.run.smithi007.stdout:5687 writesame 0x43ae3c8 thru 0x43b69e7 data_size 0x3a(0x8620 bytes) 2024-09-14T07:00:45.836 INFO:teuthology.orchestra.run.smithi007.stdout:5688 trunc from 0x6f6ae92 to 0x77f3da4 2024-09-14T07:00:45.842 INFO:teuthology.orchestra.run.smithi007.stdout:5690 write 0x5e48146 thru 0x5e51368 (0x9223 bytes) 2024-09-14T07:00:45.851 INFO:teuthology.orchestra.run.smithi007.stdout:5691 read 0x109025 thru 0x110201 (0x71dd bytes) 2024-09-14T07:00:45.857 INFO:teuthology.orchestra.run.smithi007.stdout:5693 write 0x79bfa73 thru 0x79cb691 (0xbc1f bytes) 2024-09-14T07:00:45.858 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:45.857+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:45.872 INFO:teuthology.orchestra.run.smithi007.stdout:5694 write 0x7c19582 thru 0x7c274a0 (0xdf1f bytes) 2024-09-14T07:00:45.890 INFO:teuthology.orchestra.run.smithi007.stdout:5695 write 0x9a7bf85 thru 0x9a84d15 (0x8d91 bytes) 2024-09-14T07:00:45.912 INFO:teuthology.orchestra.run.smithi007.stdout:5696 trunc from 0x9a84d16 to 0xe47f3fa 2024-09-14T07:00:45.913 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:45.913 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:45.914 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:45.914 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:45.917 INFO:teuthology.orchestra.run.smithi007.stdout:5699 write 0x5f7dafb thru 0x5f80697 (0x2b9d bytes) 2024-09-14T07:00:45.923 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:45.926 INFO:teuthology.orchestra.run.smithi007.stdout:5700 punch from 0xe27de87 to 0xe28d4d8, (0xf651 bytes) 2024-09-14T07:00:45.926 INFO:teuthology.orchestra.run.smithi007.stdout:5701 read 0x5f690c9 thru 0x5f7529a (0xc1d2 bytes) 2024-09-14T07:00:45.931 INFO:teuthology.orchestra.run.smithi007.stdout:5702 trunc from 0xe47f3fa to 0xa8ae3dc 2024-09-14T07:00:45.937 INFO:teuthology.orchestra.run.smithi007.stdout:5703 read 0x61c35f6 thru 0x61d11a3 (0xdbae bytes) 2024-09-14T07:00:45.946 INFO:teuthology.orchestra.run.smithi007.stdout:5704 writesame 0xe0e4862 thru 0xe0f1aee data_size 0x71(0xd28d bytes) 2024-09-14T07:00:45.947 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.947 INFO:teuthology.orchestra.run.smithi007.stdout:5706 trunc from 0xe0f1aef to 0xd2477fa 2024-09-14T07:00:45.959 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:45.958+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:45.960 INFO:teuthology.orchestra.run.smithi007.stdout:5707 trunc from 0xd2477fa to 0x765b69d 2024-09-14T07:00:45.976 INFO:teuthology.orchestra.run.smithi007.stdout:5709 read 0x690bd4a thru 0x6916b4b (0xae02 bytes) 2024-09-14T07:00:45.976 INFO:teuthology.orchestra.run.smithi007.stdout:5711 read 0xd245df thru 0xd2ce27 (0x8849 bytes) 2024-09-14T07:00:45.976 INFO:teuthology.orchestra.run.smithi007.stdout:5712 punch from 0x8cb663 to 0x8d9a0b, (0xe3a8 bytes) 2024-09-14T07:00:45.976 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:45.976 INFO:teuthology.orchestra.run.smithi007.stdout:5715 read 0x7c8bf3 thru 0x7d4dda (0xc1e8 bytes) 2024-09-14T07:00:45.976 INFO:teuthology.orchestra.run.smithi007.stdout:5718 writesame 0x6a15269 thru 0x6a1d85e data_size 0xd(0x85f6 bytes) 2024-09-14T07:00:45.985 INFO:teuthology.orchestra.run.smithi007.stdout:5719 punch from 0x19818f0 to 0x198469b, (0x2dab bytes) 2024-09-14T07:00:45.994 INFO:teuthology.orchestra.run.smithi007.stdout:5721 write 0x9711510 thru 0x9719892 (0x8383 bytes) 2024-09-14T07:00:46.003 INFO:teuthology.orchestra.run.smithi007.stdout:5722 read 0x2fc83d8 thru 0x2fcc391 (0x3fba bytes) 2024-09-14T07:00:46.003 INFO:teuthology.orchestra.run.smithi007.stdout:5723 write 0x37e4340 thru 0x37e4eec (0xbad bytes) 2024-09-14T07:00:46.007 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.007 INFO:teuthology.orchestra.run.smithi007.stdout:5726 write 0x63520ef thru 0x635b600 (0x9512 bytes) 2024-09-14T07:00:46.016 INFO:teuthology.orchestra.run.smithi007.stdout:5727 writesame 0x9250582 thru 0x925649b data_size 0xe(0x5f1a bytes) 2024-09-14T07:00:46.026 INFO:teuthology.orchestra.run.smithi007.stdout:5728 writesame 0x4e1b163 thru 0x4e1f33b data_size 0x3(0x41d9 bytes) 2024-09-14T07:00:46.036 INFO:teuthology.orchestra.run.smithi007.stdout:5729 punch from 0x5ef6800 to 0x5effad3, (0x92d3 bytes) 2024-09-14T07:00:46.037 INFO:teuthology.orchestra.run.smithi007.stdout:5730 write 0x6b5e0d3 thru 0x6b68918 (0xa846 bytes) 2024-09-14T07:00:46.044 INFO:teuthology.orchestra.run.smithi007.stdout:5732 writesame 0x60fb208 thru 0x61015be data_size 0x43(0x63b7 bytes) 2024-09-14T07:00:46.055 INFO:teuthology.orchestra.run.smithi007.stdout:5733 writesame 0x96f86ea thru 0x96fcf31 data_size 0x24(0x4848 bytes) 2024-09-14T07:00:46.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:46.058 INFO:teuthology.orchestra.run.smithi007.stdout:5734 write 0x3518473 thru 0x352468e (0xc21c bytes) 2024-09-14T07:00:46.060 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:46.059+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:46.068 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.068 INFO:teuthology.orchestra.run.smithi007.stdout:5736 punch from 0x6f76524 to 0x6f7a790, (0x426c bytes) 2024-09-14T07:00:46.068 INFO:teuthology.orchestra.run.smithi007.stdout:5737 read 0x26a7d44 thru 0x26adf37 (0x61f4 bytes) 2024-09-14T07:00:46.068 INFO:teuthology.orchestra.run.smithi007.stdout:5738 punch from 0x65fa7a1 to 0x660374c, (0x8fab bytes) 2024-09-14T07:00:46.068 INFO:teuthology.orchestra.run.smithi007.stdout:5739 write 0x48d5510 thru 0x48e320e (0xdcff bytes) 2024-09-14T07:00:46.076 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:46.076 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:46.076 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:46.076 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:46.076 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:46.079 INFO:teuthology.orchestra.run.smithi007.stdout:5740 write 0x1450532 thru 0x145aa7d (0xa54c bytes) 2024-09-14T07:00:46.085 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:46.090 INFO:teuthology.orchestra.run.smithi007.stdout:5741 read 0x5d31ec5 thru 0x5d4155e (0xf69a bytes) 2024-09-14T07:00:46.091 INFO:teuthology.orchestra.run.smithi007.stdout:5742 flatten 2024-09-14T07:00:46.097 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.097 INFO:teuthology.orchestra.run.smithi007.stdout:5744 punch from 0x3201863 to 0x3204393, (0x2b30 bytes) 2024-09-14T07:00:46.097 INFO:teuthology.orchestra.run.smithi007.stdout:5748 punch from 0x5d6d9f2 to 0x5d70a22, (0x3030 bytes) 2024-09-14T07:00:46.097 INFO:teuthology.orchestra.run.smithi007.stdout:5749 write 0x6fe8bb1 thru 0x6fef7a3 (0x6bf3 bytes) 2024-09-14T07:00:46.099 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.100 INFO:teuthology.orchestra.run.smithi007.stdout:5754 read 0x3a8141c thru 0x3a85829 (0x440e bytes) 2024-09-14T07:00:46.100 INFO:teuthology.orchestra.run.smithi007.stdout:5755 read 0x6205933 thru 0x62113c0 (0xba8e bytes) 2024-09-14T07:00:46.110 INFO:teuthology.orchestra.run.smithi007.stdout:5756 writesame 0xd2f8a29 thru 0xd2f8be3 data_size 0x1(0x1bb bytes) 2024-09-14T07:00:46.111 INFO:teuthology.orchestra.run.smithi007.stdout:5758 punch from 0x33ef8b6 to 0x33fba59, (0xc1a3 bytes) 2024-09-14T07:00:46.111 INFO:teuthology.orchestra.run.smithi007.stdout:5759 read 0xb3047d9 thru 0xb30d124 (0x894c bytes) 2024-09-14T07:00:46.111 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.111 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.111 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.111 INFO:teuthology.orchestra.run.smithi007.stdout:5763 read 0xcca6b05 thru 0xcca78a8 (0xda4 bytes) 2024-09-14T07:00:46.111 INFO:teuthology.orchestra.run.smithi007.stdout:5764 read 0x27e1fa6 thru 0x27ef819 (0xd874 bytes) 2024-09-14T07:00:46.118 INFO:teuthology.orchestra.run.smithi007.stdout:5766 trunc from 0xd2f8be4 to 0x647bf72 2024-09-14T07:00:46.150 INFO:teuthology.orchestra.run.smithi007.stdout:5767 writesame 0xa2962e9 thru 0xa299761 data_size 0x13(0x3479 bytes) 2024-09-14T07:00:46.150 INFO:teuthology.orchestra.run.smithi007.stdout:5768 write 0x39fa2aa thru 0x3a04d76 (0xaacd bytes) 2024-09-14T07:00:46.159 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:46.159+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:46.161 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.161 INFO:teuthology.orchestra.run.smithi007.stdout:5771 punch from 0x29cb6ed to 0x29d04bb, (0x4dce bytes) 2024-09-14T07:00:46.161 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.161 INFO:teuthology.orchestra.run.smithi007.stdout:5773 read 0x4cd390b thru 0x4cd76b2 (0x3da8 bytes) 2024-09-14T07:00:46.163 INFO:teuthology.orchestra.run.smithi007.stdout:5774 trunc from 0xa299762 to 0xc79d0ef 2024-09-14T07:00:46.169 INFO:teuthology.orchestra.run.smithi007.stdout:5777 writesame 0x62a191f thru 0x62ab669 data_size 0x1(0x9d4b bytes) 2024-09-14T07:00:46.184 INFO:teuthology.orchestra.run.smithi007.stdout:5779 write 0x36841bb thru 0x368e4fc (0xa342 bytes) 2024-09-14T07:00:46.199 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:46.199 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:46.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:46.199 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:46.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.199 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:46.199 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.032466075999999, 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:00:46.203 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.381122971000003, 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.204 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:00:46.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.208 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.381107503000003, 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:00:46.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.381096202000002, 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.213 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:00:46.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.217 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374352729000002, 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.218 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:00:46.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:00:46.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:00:46.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:00:46.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:00:46.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:00:46.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.226 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.226 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374340125000003, 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374320609000002, 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.231 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.235 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374308568000004, 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:46.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:00:46.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:00:46.239 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374288266000001, 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.240 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374275949999998, 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.244 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:46.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:00:46.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374264510000003, 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.249 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:46.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:00:46.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:00:46.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374254010999998, 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.253 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.254 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.254 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:00:46.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.254 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374243817999997, 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.258 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:46.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:00:46.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:00:46.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:00:46.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:00:46.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.262 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.263 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.263 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.263 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:46.263 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374228271, 2024-09-14T07:00:46.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:00:46.263 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:00:46.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:00:46.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374198663000001, 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:00:46.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:00:46.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:00:46.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374177013000001, 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.272 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:00:46.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:00:46.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.374154773999997, 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:00:46.276 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:00:46.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.368558409000002, 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.284 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 38.368548355999998, 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.285 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:118426 3.b 3:d50c2793:::rbd_data.114f74d20483.000000000000001c:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.697166553000002, 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.95135023100000005, 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:46.289 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 118426 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:46.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500745+0000", 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500748+0000", 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1329999999999999e-06 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:46.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500759+0000", 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0475000000000001e-05 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500765+0000", 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500824+0000", 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500830+0000", 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5889999999999999e-06 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450787+0000", 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.94995644099999998 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450901+0000", 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114032 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:46.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451052+0000", 2024-09-14T07:00:46.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015113 2024-09-14T07:00:46.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:46.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451746+0000", 2024-09-14T07:00:46.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069442799999999997 2024-09-14T07:00:46.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452006+0000", 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025975 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452059+0000", 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3090000000000002e-05 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452095+0000", 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5395999999999998e-05 2024-09-14T07:00:46.295 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.296 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.296 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.296 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:46.296 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:46.296 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:46.297 INFO:teuthology.orchestra.run.smithi007.stdout:5780 write 0xd21469f thru 0xd2230da (0xea3c bytes) 2024-09-14T07:00:46.297 INFO:teuthology.orchestra.run.smithi007.stdout:5781 read 0x4e27835 thru 0x4e28056 (0x822 bytes) 2024-09-14T07:00:46.297 INFO:teuthology.orchestra.run.smithi007.stdout:5782 write 0x7216235 thru 0x7216cb5 (0xa81 bytes) 2024-09-14T07:00:46.298 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:46.298 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:46.299 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:46.299 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.299 INFO:teuthology.orchestra.run.smithi007.stdout:5785 write 0xe476b2 thru 0xe48e5c (0x17ab bytes) 2024-09-14T07:00:46.300 INFO:teuthology.orchestra.run.smithi007.stdout:5787 read 0x1ed105e thru 0x1ed99c3 (0x8966 bytes) 2024-09-14T07:00:46.300 INFO:teuthology.orchestra.run.smithi007.stdout:5788 writesame 0x843fb7e thru 0x844b8ed data_size 0x38(0xbd70 bytes) 2024-09-14T07:00:46.300 INFO:teuthology.orchestra.run.smithi007.stdout:5790 clone 31 order 18 su 4096 sc 13 2024-09-14T07:00:46.300 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:46.300 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:46.301 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:46.302 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:46.302 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:46.302 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:46.296+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:46.304 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:46.304 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:46.321 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:46.321 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:46.337 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:46.338 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:46.398 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:46.397+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:46.442 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:46.442 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:46.451 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:46.498 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:46.498+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:46.555 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:46.555 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:46.563 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:46.599 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:46.598+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:46.670 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:46.670 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:46.678 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:46.700 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:46.700+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:46.761 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-14T07:00:46.784 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:46.784 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:46.793 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:46.831 INFO:teuthology.orchestra.run.smithi007.stdout:5791 writesame 0xc02acd7 thru 0xc039c56 data_size 0x40(0xef80 bytes) 2024-09-14T07:00:46.837 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:46.837+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:46.848 INFO:teuthology.orchestra.run.smithi007.stdout:5792 trunc from 0xd2230db to 0xd223a76 2024-09-14T07:00:46.853 INFO:teuthology.orchestra.run.smithi007.stdout:5794 write 0x37542c9 thru 0x375ccfd (0x8a35 bytes) 2024-09-14T07:00:46.889 INFO:teuthology.orchestra.run.smithi007.stdout:5795 writesame 0xd94ed5c thru 0xd9596ac data_size 0x1(0xa951 bytes) 2024-09-14T07:00:46.889 INFO:teuthology.orchestra.run.smithi007.stdout:5796 writesame 0x14ce020 thru 0x14d6182 data_size 0x3d(0x8163 bytes) 2024-09-14T07:00:46.902 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:46.901+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:46.916 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:46.916 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:46.919 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.919 INFO:teuthology.orchestra.run.smithi007.stdout:5798 read 0x2ac29dd thru 0x2ace185 (0xb7a9 bytes) 2024-09-14T07:00:46.919 INFO:teuthology.orchestra.run.smithi007.stdout:5799 writesame 0x58c5c26 thru 0x58ceba9 data_size 0x37(0x8f84 bytes) 2024-09-14T07:00:46.926 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:46.936 INFO:teuthology.orchestra.run.smithi007.stdout:5800 write 0xdcaf8c9 thru 0xdcb38d1 (0x4009 bytes) 2024-09-14T07:00:46.942 INFO:teuthology.orchestra.run.smithi007.stdout:5801 writesame 0x572e622 thru 0x5739838 data_size 0x5b(0xb217 bytes) 2024-09-14T07:00:46.968 INFO:teuthology.orchestra.run.smithi007.stdout:5802 read 0x8cc47d5 thru 0x8cd2a6d (0xe299 bytes) 2024-09-14T07:00:46.968 INFO:teuthology.orchestra.run.smithi007.stdout:5803 writesame 0xa845c60 thru 0xa85230f data_size 0x88(0xc6b0 bytes) 2024-09-14T07:00:46.983 INFO:teuthology.orchestra.run.smithi007.stdout:5804 write 0xc6b23e1 thru 0xc6b43f1 (0x2011 bytes) 2024-09-14T07:00:46.988 INFO:teuthology.orchestra.run.smithi007.stdout:5805 punch from 0x16d16e7 to 0x16db164, (0x9a7d bytes) 2024-09-14T07:00:46.988 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.988 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.988 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:46.988 INFO:teuthology.orchestra.run.smithi007.stdout:5809 clone 32 order 24 su 4194304 sc 15 2024-09-14T07:00:47.012 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:47.011+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:47.073 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:47.073 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:47.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:47.123 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:47.122+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:47.189 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:47.189 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:47.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:47.242 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:47.241+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:47.305 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:47.305 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:47.305 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:47.304+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:47.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:47.407 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:47.406+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:47.461 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:47.462 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:47.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:47.508 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:47.507+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:47.580 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:47.580 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:47.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:00:47.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:47.609 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:47.608+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:47.633 INFO:teuthology.orchestra.run.smithi007.stdout:truncating image image_client.0-clone31 from 0xdcb38d2 (overlap 0xd2230db) to 0xa81c0af 2024-09-14T07:00:47.709 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:47.708+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:47.789 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:47.790 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:47.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:47.810 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:47.810+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:47.860 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:47.860 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:47.860 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T07:00:47.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:00:47.910 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-14T07:00:47.913 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:47.911+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:47.958 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:47.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:48.017 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:48.016+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:48.113 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:48.112+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:48.130 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:48.130 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:48.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:48.189 INFO:teuthology.orchestra.run.smithi007.stdout:5811 read 0x4a6d195 thru 0x4a6e768 (0x15d4 bytes) 2024-09-14T07:00:48.191 INFO:teuthology.orchestra.run.smithi007.stdout:5813 read 0x9e124fa thru 0x9e1a1f7 (0x7cfe bytes) 2024-09-14T07:00:48.191 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.191 INFO:teuthology.orchestra.run.smithi007.stdout:5819 writesame 0x6e2d5b0 thru 0x6e3396c data_size 0x3(0x63bd bytes) 2024-09-14T07:00:48.213 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:48.213+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 91, 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6", 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-14T06:58:59.479055+0000", 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-14T07:00:47.568941+0000", 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-14T06:59:11.718934+0000", 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-14T07:00:26.509032+0000", 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5804032, 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-14T07:00:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 6, 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-14T07:00:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:11.765419+0000", 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:00:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-14T07:00:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "18", 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:00:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:00:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:00:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-14T07:00:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-14T07:00:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:00:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:00:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:00:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:00:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:00:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:00:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 6.9000000953674316, 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 6.9000000953674316, 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-14T07:00:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:18.343475+0000", 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:00:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:00:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "22", 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 22, 2024-09-14T07:00:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:00:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:00:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3.5, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3.5, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 3.5, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 3.5, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T07:00:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:27.257805+0000", 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:00:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:00:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:00:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:00:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:00:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:00:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:00:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-14T07:00:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-14T07:00:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-14T07:00:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "91", 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:00:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 34, 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 91, 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:00:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:00:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.5299999713897705, 2024-09-14T07:00:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.5299999713897705, 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.875, 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T07:00:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c036038d-cffb-4584-8851-69604a31ee40", 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 82, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:00:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/2081065366", 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/2081065366", 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/2081065366", 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/2081065366", 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "1c80080f-932a-4957-b64b-f30c4bdb9340", 2024-09-14T07:00:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 79, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6806/1688050909", 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6810/1688050909", 2024-09-14T07:00:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/1688050909", 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/1688050909", 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f3544d11-8a0f-498c-a03c-5d34134c9773", 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 82, 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-14T07:00:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-14T07:00:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/1872259061", 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/1872259061", 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/1872259061", 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/1872259061", 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d2261455-c91b-414b-925f-388f0e094dbe", 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.6944732666015625, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 77, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/1193301030", 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/1193301030", 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6811/1193301030", 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6807/1193301030", 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4088d23b-0d06-4415-9eba-6fb64355746c", 2024-09-14T07:00:48.260 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 76, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.261 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6810", 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6811", 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6812", 2024-09-14T07:00:48.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6813", 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6816", 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6817", 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.264 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6814", 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6815", 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6811/2283562461", 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6813/2283562461", 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6817/2283562461", 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6815/2283562461", 2024-09-14T07:00:48.265 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "5e27cb53-ecce-4061-bc5a-be599bf18cc4", 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:48.266 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 77, 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6818", 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6819", 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:48.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6820", 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6821", 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:48.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6824", 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6825", 2024-09-14T07:00:48.269 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6822", 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.270 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6823", 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6819/2212769626", 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6821/2212769626", 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6825/2212769626", 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6823/2212769626", 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ceb9a1f1-f54c-48d5-91d8-2287cac96ceb", 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:48.272 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6826", 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6827", 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6828", 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6829", 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6832", 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6833", 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6830", 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6831", 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.276 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6827/3197231389", 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6829/3197231389", 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6833/3197231389", 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6831/3197231389", 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85c59cd6-6e2b-4cc0-852d-d1f075a8826d", 2024-09-14T07:00:48.277 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 82, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6800", 2024-09-14T07:00:48.278 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6801", 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6802", 2024-09-14T07:00:48.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6803", 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6806", 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:48.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6807", 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6804", 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:48.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6805", 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6801/2162534312", 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6803/2162534312", 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6807/2162534312", 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6805/2162534312", 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:00:48.282 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.064941+0000", 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:48.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.780598+0000", 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:48.284 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.127700+0000", 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:48.285 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.730995+0000", 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.864423+0000", 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T07:00:48.286 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.329361+0000", 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:48.287 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:10.107581+0000", 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.300043+0000", 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:48.288 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.b", 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.289 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 7, 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1f", 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 5, 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 0 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.290 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:48.292 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:48.294 INFO:tasks.thrashosds.thrasher:j is {'epoch': 91, 'fsid': '32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6', 'created': '2024-09-14T06:58:59.479055+0000', 'modified': '2024-09-14T07:00:47.568941+0000', 'last_up_change': '2024-09-14T06:59:11.718934+0000', 'last_in_change': '2024-09-14T07:00:26.509032+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:59:11.765419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:59:18.343475+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.5, 'score_stable': 3.5, 'optimal_score': 1, 'raw_score_acting': 3.5, 'raw_score_stable': 3.5, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:59:27.257805+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 34, 'snap_epoch': 91, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5299999713897705, 'score_stable': 1.5299999713897705, 'optimal_score': 1, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c036038d-cffb-4584-8851-69604a31ee40', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 2081065366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 2081065366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 2081065366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 2081065366}]}, 'public_addr': '172.21.15.7:6817/2081065366', 'cluster_addr': '172.21.15.7:6819/2081065366', 'heartbeat_back_addr': '172.21.15.7:6823/2081065366', 'heartbeat_front_addr': '172.21.15.7:6821/2081065366', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1c80080f-932a-4957-b64b-f30c4bdb9340', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 79, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6806', 'nonce': 1688050909}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6810', 'nonce': 1688050909}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 1688050909}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 1688050909}]}, 'public_addr': '172.21.15.7:6806/1688050909', 'cluster_addr': '172.21.15.7:6810/1688050909', 'heartbeat_back_addr': '172.21.15.7:6815/1688050909', 'heartbeat_front_addr': '172.21.15.7:6813/1688050909', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f3544d11-8a0f-498c-a03c-5d34134c9773', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 1872259061}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 1872259061}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 1872259061}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 1872259061}]}, 'public_addr': '172.21.15.7:6825/1872259061', 'cluster_addr': '172.21.15.7:6827/1872259061', 'heartbeat_back_addr': '172.21.15.7:6831/1872259061', 'heartbeat_front_addr': '172.21.15.7:6829/1872259061', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd2261455-c91b-414b-925f-388f0e094dbe', 'up': 1, 'in': 1, 'weight': 0.6944732666015625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 1193301030}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 1193301030}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6809', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 1193301030}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6805', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 1193301030}]}, 'public_addr': '172.21.15.7:6801/1193301030', 'cluster_addr': '172.21.15.7:6803/1193301030', 'heartbeat_back_addr': '172.21.15.7:6811/1193301030', 'heartbeat_front_addr': '172.21.15.7:6807/1193301030', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4088d23b-0d06-4415-9eba-6fb64355746c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6810', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6811', 'nonce': 2283562461}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6812', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6813', 'nonce': 2283562461}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6816', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6817', 'nonce': 2283562461}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6814', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6815', 'nonce': 2283562461}]}, 'public_addr': '172.21.15.189:6811/2283562461', 'cluster_addr': '172.21.15.189:6813/2283562461', 'heartbeat_back_addr': '172.21.15.189:6817/2283562461', 'heartbeat_front_addr': '172.21.15.189:6815/2283562461', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5e27cb53-ecce-4061-bc5a-be599bf18cc4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6818', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6819', 'nonce': 2212769626}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6820', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6821', 'nonce': 2212769626}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6824', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6825', 'nonce': 2212769626}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6822', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6823', 'nonce': 2212769626}]}, 'public_addr': '172.21.15.189:6819/2212769626', 'cluster_addr': '172.21.15.189:6821/2212769626', 'heartbeat_back_addr': '172.21.15.189:6825/2212769626', 'heartbeat_front_addr': '172.21.15.189:6823/2212769626', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceb9a1f1-f54c-48d5-91d8-2287cac96ceb', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6826', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6827', 'nonce': 3197231389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6828', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6829', 'nonce': 3197231389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 3197231389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6830', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6831', 'nonce': 3197231389}]}, 'public_addr': '172.21.15.189:6827/3197231389', 'cluster_addr': '172.21.15.189:6829/3197231389', 'heartbeat_back_addr': '172.21.15.189:6833/3197231389', 'heartbeat_front_addr': '172.21.15.189:6831/3197231389', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '85c59cd6-6e2b-4cc0-852d-d1f075a8826d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6800', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6801', 'nonce': 2162534312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6802', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6803', 'nonce': 2162534312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6806', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6807', 'nonce': 2162534312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6804', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6805', 'nonce': 2162534312}]}, 'public_addr': '172.21.15.189:6801/2162534312', 'cluster_addr': '172.21.15.189:6803/2162534312', 'heartbeat_back_addr': '172.21.15.189:6807/2162534312', 'heartbeat_front_addr': '172.21.15.189:6805/2162534312', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.064941+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.780598+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.127700+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.730995+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.864423+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.329361+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T06:59:10.107581+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.300043+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.18', 'osds': [0, 5]}], 'pg_upmap_items': [{'pgid': '3.b', 'mappings': [{'from': 7, 'to': 5}]}, {'pgid': '3.1f', 'mappings': [{'from': 5, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:00:48.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:00:48.295 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.295 INFO:teuthology.orchestra.run.smithi007.stdout:5822 trunc from 0xdcb38d2 to 0x15f4ce6 2024-09-14T07:00:48.314 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:48.314 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:48.318 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:48.313+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:48.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:48.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:48.415 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:48.414+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:48.525 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:48.515+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:48.567 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:48.568 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:48.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:48.587 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T07:00:48.616 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:48.615+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:48.621 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:48.621 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:48.657 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:48.691 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:48.717 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:48.717+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:48.726 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:48.726 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:48.744 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:48.745 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:48.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:48.761 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:48.762 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:48.794 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:48.794 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:48.818 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:48.817+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:48.821 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:48.821 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:48.822 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.822 INFO:teuthology.orchestra.run.smithi007.stdout:5826 read 0x79fff4 thru 0x7ab95b (0xb968 bytes) 2024-09-14T07:00:48.822 INFO:teuthology.orchestra.run.smithi007.stdout:5827 read 0x1414da2 thru 0x142370e (0xe96d bytes) 2024-09-14T07:00:48.831 INFO:teuthology.orchestra.run.smithi007.stdout:5828 write 0x2a2d537 thru 0x2a35499 (0x7f63 bytes) 2024-09-14T07:00:48.839 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:48.840 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:48.845 INFO:teuthology.orchestra.run.smithi007.stdout:5830 write 0x59fce2b thru 0x5a04644 (0x781a bytes) 2024-09-14T07:00:48.850 INFO:teuthology.orchestra.run.smithi007.stdout:5831 punch from 0xba3753 to 0xbb2fa8, (0xf855 bytes) 2024-09-14T07:00:48.853 INFO:teuthology.orchestra.run.smithi007.stdout:5832 trunc from 0x5a04645 to 0x3546a81 2024-09-14T07:00:48.877 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:00:48.878 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:00:48.879 INFO:teuthology.orchestra.run.smithi007.stdout:5833 trunc from 0x3546a81 to 0xe608653 2024-09-14T07:00:48.883 INFO:teuthology.orchestra.run.smithi007.stdout:5834 write 0xe039de2 thru 0xe042aad (0x8ccc bytes) 2024-09-14T07:00:48.889 INFO:teuthology.orchestra.run.smithi007.stdout:5835 read 0xa298602 thru 0xa29da6c (0x546b bytes) 2024-09-14T07:00:48.889 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.891 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-14T07:00:48.583717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":909233729,"num_objects":2016,"num_object_clones":519,"num_object_copies":4032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2016,"num_whiteouts":881,"num_read":197840,"num_read_kb":439788,"num_write":19755,"num_write_kb":231350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1957,"num_bytes_recovered":79804443,"num_keys_recovered":223,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16161,"ondisk_log_size":16161,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":406892,"kb_used_data":41404,"kb_used_omap":10,"kb_used_meta":365301,"kb_avail":660195988,"statfs":{"total":676457349120,"available":676040691712,"internally_reserved":0,"allocated":42397696,"data_stored":271784501,"data_compressed":8525384,"data_compressed_allocated":21483520,"data_compressed_original":261010997,"omap_allocated":11157,"internal_metadata":374068331},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23135798,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":18185,"num_read_kb":25362,"num_write":4521,"num_write_kb":11763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.819015"},"pg_stats":[{"pgid":"3.1f","version":"90'364","reported_seq":6629,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.981590+0000","last_change":"2024-09-14T07:00:01.385727+0000","last_active":"2024-09-14T07:00:46.981590+0000","last_peered":"2024-09-14T07:00:46.981590+0000","last_clean":"2024-09-14T07:00:46.981590+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:00:46.981590+0000","last_undegraded":"2024-09-14T07:00:46.981590+0000","last_fullsized":"2024-09-14T07:00:46.981590+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:07:01.095243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043615199999999998,"stat_sum":{"num_bytes":39831467,"num_objects":73,"num_object_clones":22,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":26,"num_read":7122,"num_read_kb":26426,"num_write":347,"num_write_kb":16621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"88'330","reported_seq":2029,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T07:00:45.422060+0000","last_change":"2024-09-14T07:00:30.291631+0000","last_active":"2024-09-14T07:00:45.422060+0000","last_peered":"2024-09-14T07:00:45.422060+0000","last_clean":"2024-09-14T07:00:45.422060+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:00:45.422060+0000","last_undegraded":"2024-09-14T07:00:45.422060+0000","last_fullsized":"2024-09-14T07:00:45.422060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:30:17.168605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000412824,"stat_sum":{"num_bytes":19337095,"num_objects":61,"num_object_clones":16,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":32,"num_read":1996,"num_read_kb":7589,"num_write":304,"num_write_kb":7157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"90'258","reported_seq":5040,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.916410+0000","last_change":"2024-09-14T07:00:36.548711+0000","last_active":"2024-09-14T07:00:46.916410+0000","last_peered":"2024-09-14T07:00:46.916410+0000","last_clean":"2024-09-14T07:00:46.916410+0000","last_became_active":"2024-09-14T07:00:36.547965+0000","last_became_peered":"2024-09-14T07:00:36.547965+0000","last_unstale":"2024-09-14T07:00:46.916410+0000","last_undegraded":"2024-09-14T07:00:46.916410+0000","last_fullsized":"2024-09-14T07:00:46.916410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:46:03.869231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001689915,"stat_sum":{"num_bytes":11525040,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":31,"num_read":4706,"num_read_kb":12298,"num_write":255,"num_write_kb":5058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"90'1910","reported_seq":2778,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.878344+0000","last_change":"2024-09-14T06:59:30.806817+0000","last_active":"2024-09-14T07:00:46.878344+0000","last_peered":"2024-09-14T07:00:46.878344+0000","last_clean":"2024-09-14T07:00:46.878344+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:00:46.878344+0000","last_undegraded":"2024-09-14T07:00:46.878344+0000","last_fullsized":"2024-09-14T07:00:46.878344+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1910,"log_dups_size":0,"ondisk_log_size":1910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:54:46.951928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042050300000000002,"stat_sum":{"num_bytes":11798444,"num_objects":71,"num_object_clones":19,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":30,"num_read":9882,"num_read_kb":12974,"num_write":3323,"num_write_kb":7075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"89'773","reported_seq":1582,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.180827+0000","last_change":"2024-09-14T06:59:30.801680+0000","last_active":"2024-09-14T07:00:46.180827+0000","last_peered":"2024-09-14T07:00:46.180827+0000","last_clean":"2024-09-14T07:00:46.180827+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:00:46.180827+0000","last_undegraded":"2024-09-14T07:00:46.180827+0000","last_fullsized":"2024-09-14T07:00:46.180827+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:51:35.681692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060885900000000001,"stat_sum":{"num_bytes":52850307,"num_objects":74,"num_object_clones":20,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":33,"num_read":2981,"num_read_kb":10383,"num_write":1057,"num_write_kb":8532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"90'1035","reported_seq":6910,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.936175+0000","last_change":"2024-09-14T06:59:30.807597+0000","last_active":"2024-09-14T07:00:46.936175+0000","last_peered":"2024-09-14T07:00:46.936175+0000","last_clean":"2024-09-14T07:00:46.936175+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:00:46.936175+0000","last_undegraded":"2024-09-14T07:00:46.936175+0000","last_fullsized":"2024-09-14T07:00:46.936175+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1035,"log_dups_size":0,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:54:47.600673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010420550000000001,"stat_sum":{"num_bytes":26436250,"num_objects":69,"num_object_clones":14,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":26,"num_read":11054,"num_read_kb":18329,"num_write":1537,"num_write_kb":6616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"89'444","reported_seq":3807,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:00:45.813632+0000","last_change":"2024-09-14T06:59:30.801946+0000","last_active":"2024-09-14T07:00:45.813632+0000","last_peered":"2024-09-14T07:00:45.813632+0000","last_clean":"2024-09-14T07:00:45.813632+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:00:45.813632+0000","last_undegraded":"2024-09-14T07:00:45.813632+0000","last_fullsized":"2024-09-14T07:00:45.813632+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:58:03.963439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081998900000000003,"stat_sum":{"num_bytes":67388516,"num_objects":61,"num_object_clones":15,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":27,"num_read":3867,"num_read_kb":11292,"num_write":480,"num_write_kb":11734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"90'513","reported_seq":2624,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.959082+0000","last_change":"2024-09-14T06:59:37.046075+0000","last_active":"2024-09-14T07:00:46.959082+0000","last_peered":"2024-09-14T07:00:46.959082+0000","last_clean":"2024-09-14T07:00:46.959082+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:00:46.959082+0000","last_undegraded":"2024-09-14T07:00:46.959082+0000","last_fullsized":"2024-09-14T07:00:46.959082+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:12.403199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059456000000000005,"stat_sum":{"num_bytes":20278503,"num_objects":66,"num_object_clones":16,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":28,"num_read":3916,"num_read_kb":8792,"num_write":578,"num_write_kb":5405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"88'272","reported_seq":5645,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T07:00:45.192922+0000","last_change":"2024-09-14T06:59:30.809133+0000","last_active":"2024-09-14T07:00:45.192922+0000","last_peered":"2024-09-14T07:00:45.192922+0000","last_clean":"2024-09-14T07:00:45.192922+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:00:45.192922+0000","last_undegraded":"2024-09-14T07:00:45.192922+0000","last_fullsized":"2024-09-14T07:00:45.192922+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":272,"log_dups_size":0,"ondisk_log_size":272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:29:58.346113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055309100000000002,"stat_sum":{"num_bytes":16332700,"num_objects":55,"num_object_clones":11,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":30,"num_read":6424,"num_read_kb":15046,"num_write":240,"num_write_kb":2590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"90'329","reported_seq":3857,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.968532+0000","last_change":"2024-09-14T07:00:36.548676+0000","last_active":"2024-09-14T07:00:46.968532+0000","last_peered":"2024-09-14T07:00:46.968532+0000","last_clean":"2024-09-14T07:00:46.968532+0000","last_became_active":"2024-09-14T07:00:36.547481+0000","last_became_peered":"2024-09-14T07:00:36.547481+0000","last_unstale":"2024-09-14T07:00:46.968532+0000","last_undegraded":"2024-09-14T07:00:46.968532+0000","last_fullsized":"2024-09-14T07:00:46.968532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:06:20.058340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086036299999999995,"stat_sum":{"num_bytes":9533368,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":4073,"num_read_kb":7258,"num_write":308,"num_write_kb":3730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"90'259","reported_seq":916,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.964103+0000","last_change":"2024-09-14T06:59:51.355900+0000","last_active":"2024-09-14T07:00:46.964103+0000","last_peered":"2024-09-14T07:00:46.964103+0000","last_clean":"2024-09-14T07:00:46.964103+0000","last_became_active":"2024-09-14T06:59:51.330515+0000","last_became_peered":"2024-09-14T06:59:51.330515+0000","last_unstale":"2024-09-14T07:00:46.964103+0000","last_undegraded":"2024-09-14T07:00:46.964103+0000","last_fullsized":"2024-09-14T07:00:46.964103+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:22:46.635724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052297400000000005,"stat_sum":{"num_bytes":15883771,"num_objects":73,"num_object_clones":18,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":511,"num_read_kb":2219,"num_write":266,"num_write_kb":4282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"89'684","reported_seq":4940,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.570986+0000","last_change":"2024-09-14T07:00:36.548704+0000","last_active":"2024-09-14T07:00:46.570986+0000","last_peered":"2024-09-14T07:00:46.570986+0000","last_clean":"2024-09-14T07:00:46.570986+0000","last_became_active":"2024-09-14T07:00:36.547583+0000","last_became_peered":"2024-09-14T07:00:36.547583+0000","last_unstale":"2024-09-14T07:00:46.570986+0000","last_undegraded":"2024-09-14T07:00:46.570986+0000","last_fullsized":"2024-09-14T07:00:46.570986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:58:37.108676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013224059999999999,"stat_sum":{"num_bytes":37288210,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":32,"num_read":6207,"num_read_kb":11783,"num_write":992,"num_write_kb":7094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.570491+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:00:46.570491+0000","last_peered":"2024-09-14T07:00:46.570491+0000","last_clean":"2024-09-14T07:00:46.570491+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:00:46.570491+0000","last_undegraded":"2024-09-14T07:00:46.570491+0000","last_fullsized":"2024-09-14T07:00:46.570491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"89'609","reported_seq":5340,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.125090+0000","last_change":"2024-09-14T06:59:30.801713+0000","last_active":"2024-09-14T07:00:46.125090+0000","last_peered":"2024-09-14T07:00:46.125090+0000","last_clean":"2024-09-14T07:00:46.125090+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:00:46.125090+0000","last_undegraded":"2024-09-14T07:00:46.125090+0000","last_fullsized":"2024-09-14T07:00:46.125090+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:11:52.932884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059990300000000005,"stat_sum":{"num_bytes":17768312,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":25,"num_read":6382,"num_read_kb":7943,"num_write":729,"num_write_kb":5703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:00:45.582499+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:00:45.582499+0000","last_peered":"2024-09-14T07:00:45.582499+0000","last_clean":"2024-09-14T07:00:45.582499+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:00:45.582499+0000","last_undegraded":"2024-09-14T07:00:45.582499+0000","last_fullsized":"2024-09-14T07:00:45.582499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"89'372","reported_seq":8958,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.195009+0000","last_change":"2024-09-14T06:59:51.791781+0000","last_active":"2024-09-14T07:00:46.195009+0000","last_peered":"2024-09-14T07:00:46.195009+0000","last_clean":"2024-09-14T07:00:46.195009+0000","last_became_active":"2024-09-14T06:59:51.751790+0000","last_became_peered":"2024-09-14T06:59:51.751790+0000","last_unstale":"2024-09-14T07:00:46.195009+0000","last_undegraded":"2024-09-14T07:00:46.195009+0000","last_fullsized":"2024-09-14T07:00:46.195009+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:37:11.168588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060862000000000004,"stat_sum":{"num_bytes":15151772,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":22,"num_read":10060,"num_read_kb":16165,"num_write":394,"num_write_kb":7339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.570951+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:00:46.570951+0000","last_peered":"2024-09-14T07:00:46.570951+0000","last_clean":"2024-09-14T07:00:46.570951+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:00:46.570951+0000","last_undegraded":"2024-09-14T07:00:46.570951+0000","last_fullsized":"2024-09-14T07:00:46.570951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'376","reported_seq":2178,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T07:00:45.381171+0000","last_change":"2024-09-14T07:00:31.290784+0000","last_active":"2024-09-14T07:00:45.381171+0000","last_peered":"2024-09-14T07:00:45.381171+0000","last_clean":"2024-09-14T07:00:45.381171+0000","last_became_active":"2024-09-14T07:00:31.290234+0000","last_became_peered":"2024-09-14T07:00:31.290234+0000","last_unstale":"2024-09-14T07:00:45.381171+0000","last_undegraded":"2024-09-14T07:00:45.381171+0000","last_fullsized":"2024-09-14T07:00:45.381171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:35:20.521047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21925624,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":23,"num_read":2404,"num_read_kb":2517,"num_write":336,"num_write_kb":4001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T07:00:42.583811+0000","last_change":"2024-09-14T07:00:27.521374+0000","last_active":"2024-09-14T07:00:42.583811+0000","last_peered":"2024-09-14T07:00:42.583811+0000","last_clean":"2024-09-14T07:00:42.583811+0000","last_became_active":"2024-09-14T07:00:27.521101+0000","last_became_peered":"2024-09-14T07:00:27.521101+0000","last_unstale":"2024-09-14T07:00:42.583811+0000","last_undegraded":"2024-09-14T07:00:42.583811+0000","last_fullsized":"2024-09-14T07:00:42.583811+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:09:26.896693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"90'498","reported_seq":9131,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.941643+0000","last_change":"2024-09-14T07:00:31.292180+0000","last_active":"2024-09-14T07:00:46.941643+0000","last_peered":"2024-09-14T07:00:46.941643+0000","last_clean":"2024-09-14T07:00:46.941643+0000","last_became_active":"2024-09-14T07:00:31.290358+0000","last_became_peered":"2024-09-14T07:00:31.290358+0000","last_unstale":"2024-09-14T07:00:46.941643+0000","last_undegraded":"2024-09-14T07:00:46.941643+0000","last_fullsized":"2024-09-14T07:00:46.941643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":498,"log_dups_size":0,"ondisk_log_size":498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:53:07.945560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052993500000000004,"stat_sum":{"num_bytes":54137255,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":27,"num_read":7376,"num_read_kb":11971,"num_write":557,"num_write_kb":8284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"90'580","reported_seq":6668,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.957854+0000","last_change":"2024-09-14T07:00:30.292021+0000","last_active":"2024-09-14T07:00:46.957854+0000","last_peered":"2024-09-14T07:00:46.957854+0000","last_clean":"2024-09-14T07:00:46.957854+0000","last_became_active":"2024-09-14T07:00:30.291275+0000","last_became_peered":"2024-09-14T07:00:30.291275+0000","last_unstale":"2024-09-14T07:00:46.957854+0000","last_undegraded":"2024-09-14T07:00:46.957854+0000","last_fullsized":"2024-09-14T07:00:46.957854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:59:45.845694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044476100000000001,"stat_sum":{"num_bytes":28641311,"num_objects":61,"num_object_clones":13,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":25,"num_read":7337,"num_read_kb":9355,"num_write":808,"num_write_kb":6021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":95,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.576670+0000","last_change":"2024-09-14T07:00:33.531473+0000","last_active":"2024-09-14T07:00:46.576670+0000","last_peered":"2024-09-14T07:00:46.576670+0000","last_clean":"2024-09-14T07:00:46.576670+0000","last_became_active":"2024-09-14T07:00:33.528631+0000","last_became_peered":"2024-09-14T07:00:33.528631+0000","last_unstale":"2024-09-14T07:00:46.576670+0000","last_undegraded":"2024-09-14T07:00:46.576670+0000","last_fullsized":"2024-09-14T07:00:46.576670+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:37:51.090265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"88'307","reported_seq":20792,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T07:00:45.438624+0000","last_change":"2024-09-14T06:59:30.809288+0000","last_active":"2024-09-14T07:00:45.438624+0000","last_peered":"2024-09-14T07:00:45.438624+0000","last_clean":"2024-09-14T07:00:45.438624+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:00:45.438624+0000","last_undegraded":"2024-09-14T07:00:45.438624+0000","last_fullsized":"2024-09-14T07:00:45.438624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:57:09.124662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063151899999999998,"stat_sum":{"num_bytes":23350472,"num_objects":57,"num_object_clones":18,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":21,"num_read":20930,"num_read_kb":42447,"num_write":369,"num_write_kb":16889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T07:00:42.597241+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:00:42.597241+0000","last_peered":"2024-09-14T07:00:42.597241+0000","last_clean":"2024-09-14T07:00:42.597241+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:00:42.597241+0000","last_undegraded":"2024-09-14T07:00:42.597241+0000","last_fullsized":"2024-09-14T07:00:42.597241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'295","reported_seq":1653,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.570765+0000","last_change":"2024-09-14T07:00:30.291035+0000","last_active":"2024-09-14T07:00:46.570765+0000","last_peered":"2024-09-14T07:00:46.570765+0000","last_clean":"2024-09-14T07:00:46.570765+0000","last_became_active":"2024-09-14T07:00:30.290337+0000","last_became_peered":"2024-09-14T07:00:30.290337+0000","last_unstale":"2024-09-14T07:00:46.570765+0000","last_undegraded":"2024-09-14T07:00:46.570765+0000","last_fullsized":"2024-09-14T07:00:46.570765+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:06:29.147149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11767498,"num_objects":55,"num_object_clones":16,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":31,"num_read":2203,"num_read_kb":3717,"num_write":253,"num_write_kb":1991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.570327+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T07:00:46.570327+0000","last_peered":"2024-09-14T07:00:46.570327+0000","last_clean":"2024-09-14T07:00:46.570327+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:00:46.570327+0000","last_undegraded":"2024-09-14T07:00:46.570327+0000","last_fullsized":"2024-09-14T07:00:46.570327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"89'463","reported_seq":8418,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.064801+0000","last_change":"2024-09-14T07:00:30.291751+0000","last_active":"2024-09-14T07:00:46.064801+0000","last_peered":"2024-09-14T07:00:46.064801+0000","last_clean":"2024-09-14T07:00:46.064801+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:00:46.064801+0000","last_undegraded":"2024-09-14T07:00:46.064801+0000","last_fullsized":"2024-09-14T07:00:46.064801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:13:13.305812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037135599999999999,"stat_sum":{"num_bytes":46052570,"num_objects":63,"num_object_clones":11,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":25,"num_read":8703,"num_read_kb":33738,"num_write":631,"num_write_kb":12342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":177,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.570396+0000","last_change":"2024-09-14T07:00:27.523385+0000","last_active":"2024-09-14T07:00:46.570396+0000","last_peered":"2024-09-14T07:00:46.570396+0000","last_clean":"2024-09-14T07:00:46.570396+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:00:46.570396+0000","last_undegraded":"2024-09-14T07:00:46.570396+0000","last_fullsized":"2024-09-14T07:00:46.570396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:19.504174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":246,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.576650+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T07:00:46.576650+0000","last_peered":"2024-09-14T07:00:46.576650+0000","last_clean":"2024-09-14T07:00:46.576650+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:00:46.576650+0000","last_undegraded":"2024-09-14T07:00:46.576650+0000","last_fullsized":"2024-09-14T07:00:46.576650+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"89'460","reported_seq":8631,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.177750+0000","last_change":"2024-09-14T06:59:30.801426+0000","last_active":"2024-09-14T07:00:46.177750+0000","last_peered":"2024-09-14T07:00:46.177750+0000","last_clean":"2024-09-14T07:00:46.177750+0000","last_became_active":"2024-09-14T06:59:28.808414+0000","last_became_peered":"2024-09-14T06:59:28.808414+0000","last_unstale":"2024-09-14T07:00:46.177750+0000","last_undegraded":"2024-09-14T07:00:46.177750+0000","last_fullsized":"2024-09-14T07:00:46.177750+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:57:38.500852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038565499999999999,"stat_sum":{"num_bytes":44120334,"num_objects":75,"num_object_clones":22,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":26,"num_read":8625,"num_read_kb":19501,"num_write":462,"num_write_kb":10392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"90'572","reported_seq":3891,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.873959+0000","last_change":"2024-09-14T06:59:30.802594+0000","last_active":"2024-09-14T07:00:46.873959+0000","last_peered":"2024-09-14T07:00:46.873959+0000","last_clean":"2024-09-14T07:00:46.873959+0000","last_became_active":"2024-09-14T06:59:28.804973+0000","last_became_peered":"2024-09-14T06:59:28.804973+0000","last_unstale":"2024-09-14T07:00:46.873959+0000","last_undegraded":"2024-09-14T07:00:46.873959+0000","last_fullsized":"2024-09-14T07:00:46.873959+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:16:14.000834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010793000000000001,"stat_sum":{"num_bytes":13378642,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":23,"num_read":6374,"num_read_kb":14417,"num_write":679,"num_write_kb":6876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"90'985","reported_seq":6716,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.982276+0000","last_change":"2024-09-14T06:59:30.802721+0000","last_active":"2024-09-14T07:00:46.982276+0000","last_peered":"2024-09-14T07:00:46.982276+0000","last_clean":"2024-09-14T07:00:46.982276+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:00:46.982276+0000","last_undegraded":"2024-09-14T07:00:46.982276+0000","last_fullsized":"2024-09-14T07:00:46.982276+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:17:16.695389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001402691,"stat_sum":{"num_bytes":15259795,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":29,"num_read":9119,"num_read_kb":8195,"num_write":1441,"num_write_kb":3534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"88'269","reported_seq":1943,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T07:00:45.420067+0000","last_change":"2024-09-14T07:00:13.451566+0000","last_active":"2024-09-14T07:00:45.420067+0000","last_peered":"2024-09-14T07:00:45.420067+0000","last_clean":"2024-09-14T07:00:45.420067+0000","last_became_active":"2024-09-14T07:00:13.450417+0000","last_became_peered":"2024-09-14T07:00:13.450417+0000","last_unstale":"2024-09-14T07:00:45.420067+0000","last_undegraded":"2024-09-14T07:00:45.420067+0000","last_fullsized":"2024-09-14T07:00:45.420067+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:40.888540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056983800000000003,"stat_sum":{"num_bytes":38128595,"num_objects":53,"num_object_clones":16,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":26,"num_read":1320,"num_read_kb":6399,"num_write":240,"num_write_kb":2760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'377","reported_seq":4138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.774576+0000","last_change":"2024-09-14T06:59:30.811244+0000","last_active":"2024-09-14T07:00:46.774576+0000","last_peered":"2024-09-14T07:00:46.774576+0000","last_clean":"2024-09-14T07:00:46.774576+0000","last_became_active":"2024-09-14T06:59:28.799619+0000","last_became_peered":"2024-09-14T06:59:28.799619+0000","last_unstale":"2024-09-14T07:00:46.774576+0000","last_undegraded":"2024-09-14T07:00:46.774576+0000","last_fullsized":"2024-09-14T07:00:46.774576+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:14:18.014948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077164499999999995,"stat_sum":{"num_bytes":39968347,"num_objects":71,"num_object_clones":21,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":30,"num_read":3803,"num_read_kb":10136,"num_write":359,"num_write_kb":8427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"90'345","reported_seq":791,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.912694+0000","last_change":"2024-09-14T06:59:30.803608+0000","last_active":"2024-09-14T07:00:46.912694+0000","last_peered":"2024-09-14T07:00:46.912694+0000","last_clean":"2024-09-14T07:00:46.912694+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:00:46.912694+0000","last_undegraded":"2024-09-14T07:00:46.912694+0000","last_fullsized":"2024-09-14T07:00:46.912694+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:22:19.887337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016847640000000001,"stat_sum":{"num_bytes":19422716,"num_objects":70,"num_object_clones":17,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":30,"num_read":776,"num_read_kb":5048,"num_write":389,"num_write_kb":6052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"90'288","reported_seq":11059,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.906393+0000","last_change":"2024-09-14T07:00:31.293997+0000","last_active":"2024-09-14T07:00:46.906393+0000","last_peered":"2024-09-14T07:00:46.906393+0000","last_clean":"2024-09-14T07:00:46.906393+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:00:46.906393+0000","last_undegraded":"2024-09-14T07:00:46.906393+0000","last_fullsized":"2024-09-14T07:00:46.906393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":288,"log_dups_size":0,"ondisk_log_size":288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:35:17.056052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084307800000000003,"stat_sum":{"num_bytes":48436143,"num_objects":64,"num_object_clones":19,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":10806,"num_read_kb":43696,"num_write":277,"num_write_kb":13960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"90'407","reported_seq":3959,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.924695+0000","last_change":"2024-09-14T06:59:30.811224+0000","last_active":"2024-09-14T07:00:46.924695+0000","last_peered":"2024-09-14T07:00:46.924695+0000","last_clean":"2024-09-14T07:00:46.924695+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:00:46.924695+0000","last_undegraded":"2024-09-14T07:00:46.924695+0000","last_fullsized":"2024-09-14T07:00:46.924695+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:04:40.237941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053083799999999995,"stat_sum":{"num_bytes":10221157,"num_objects":72,"num_object_clones":17,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":36,"num_read":5067,"num_read_kb":9848,"num_write":374,"num_write_kb":4394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"90'360","reported_seq":2961,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.576862+0000","last_change":"2024-09-14T06:59:30.806812+0000","last_active":"2024-09-14T07:00:46.576862+0000","last_peered":"2024-09-14T07:00:46.576862+0000","last_clean":"2024-09-14T07:00:46.576862+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:00:46.576862+0000","last_undegraded":"2024-09-14T07:00:46.576862+0000","last_fullsized":"2024-09-14T07:00:46.576862+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:56:03.103892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055264400000000003,"stat_sum":{"num_bytes":17671701,"num_objects":56,"num_object_clones":16,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":22,"num_read":2827,"num_read_kb":11368,"num_write":366,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"90'450","reported_seq":9746,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.848016+0000","last_change":"2024-09-14T07:00:31.291669+0000","last_active":"2024-09-14T07:00:46.848016+0000","last_peered":"2024-09-14T07:00:46.848016+0000","last_clean":"2024-09-14T07:00:46.848016+0000","last_became_active":"2024-09-14T07:00:31.290804+0000","last_became_peered":"2024-09-14T07:00:31.290804+0000","last_unstale":"2024-09-14T07:00:46.848016+0000","last_undegraded":"2024-09-14T07:00:46.848016+0000","last_fullsized":"2024-09-14T07:00:46.848016+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:14:21.627660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010054669999999999,"stat_sum":{"num_bytes":19440046,"num_objects":62,"num_object_clones":17,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":27,"num_read":10674,"num_read_kb":22299,"num_write":533,"num_write_kb":7965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"88'453","reported_seq":4664,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-14T07:00:44.504384+0000","last_change":"2024-09-14T07:00:36.549023+0000","last_active":"2024-09-14T07:00:44.504384+0000","last_peered":"2024-09-14T07:00:44.504384+0000","last_clean":"2024-09-14T07:00:44.504384+0000","last_became_active":"2024-09-14T07:00:36.547241+0000","last_became_peered":"2024-09-14T07:00:36.547241+0000","last_unstale":"2024-09-14T07:00:44.504384+0000","last_undegraded":"2024-09-14T07:00:44.504384+0000","last_fullsized":"2024-09-14T07:00:44.504384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:34:50.717660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000465725,"stat_sum":{"num_bytes":39233181,"num_objects":50,"num_object_clones":14,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":25,"num_read":5377,"num_read_kb":10137,"num_write":400,"num_write_kb":4741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"90'449","reported_seq":4529,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-14T07:00:46.907918+0000","last_change":"2024-09-14T06:59:30.802121+0000","last_active":"2024-09-14T07:00:46.907918+0000","last_peered":"2024-09-14T07:00:46.907918+0000","last_clean":"2024-09-14T07:00:46.907918+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:00:46.907918+0000","last_undegraded":"2024-09-14T07:00:46.907918+0000","last_fullsized":"2024-09-14T07:00:46.907918+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:04:28.989188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087321199999999995,"stat_sum":{"num_bytes":56084200,"num_objects":74,"num_object_clones":24,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":29,"num_read":4872,"num_read_kb":6447,"num_write":368,"num_write_kb":6004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":908643342,"num_objects":2012,"num_object_clones":519,"num_object_copies":4024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2012,"num_whiteouts":881,"num_read":197774,"num_read_kb":439733,"num_write":19652,"num_write_kb":229720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1954,"num_bytes_recovered":79804424,"num_keys_recovered":223,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":35450880,"data_stored":268825568,"data_compressed":8597016,"data_compressed_allocated":21594112,"data_compressed_original":261203509,"omap_allocated":0,"internal_metadata":0},"log_size":16086,"ondisk_log_size":16086,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":0,"seq":64424509463,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53904,"kb_used_data":4612,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94317936,"statfs":{"total":96636764160,"available":96581566464,"internally_reserved":0,"allocated":4722688,"data_stored":25725418,"data_compressed":714187,"data_compressed_allocated":2027520,"data_compressed_original":24390035,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59648,"kb_used_data":6256,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94312192,"statfs":{"total":96636764160,"available":96575684608,"internally_reserved":0,"allocated":6406144,"data_stored":37589854,"data_compressed":1138485,"data_compressed_allocated":3117056,"data_compressed_original":35852166,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82008,"kb_used_data":6408,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94289832,"statfs":{"total":96636764160,"available":96552787968,"internally_reserved":0,"allocated":6561792,"data_stored":51676034,"data_compressed":1443062,"data_compressed_allocated":3670016,"data_compressed_original":50238423,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509462,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84244,"kb_used_data":8640,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94287596,"statfs":{"total":96636764160,"available":96550498304,"internally_reserved":0,"allocated":8847360,"data_stored":48827627,"data_compressed":1789448,"data_compressed_allocated":4403200,"data_compressed_original":46298826,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":5260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":5386240,"data_stored":39728797,"data_compressed":1237663,"data_compressed_allocated":2867200,"data_compressed_original":38503433,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542166,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55704,"kb_used_data":6472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94316136,"statfs":{"total":96636764160,"available":96579723264,"internally_reserved":0,"allocated":6627328,"data_stored":43676802,"data_compressed":1416998,"data_compressed_allocated":3502080,"data_compressed_original":42148296,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39100,"kb_used_data":3756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332740,"statfs":{"total":96636764160,"available":96596725760,"internally_reserved":0,"allocated":3846144,"data_stored":24559969,"data_compressed":785541,"data_compressed_allocated":1896448,"data_compressed_original":23579818,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7790592,"data_stored":48336982,"data_compressed":1789448,"data_compressed_allocated":4403200,"data_compressed_original":46298826,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5611520,"data_stored":51142518,"data_compressed":1447029,"data_compressed_allocated":3670016,"data_compressed_original":50090967,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5349376,"data_stored":37099209,"data_compressed":1138485,"data_compressed_allocated":3117056,"data_compressed_original":35852166,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3870720,"data_stored":26035823,"data_compressed":738120,"data_compressed_allocated":2101248,"data_compressed_original":25078163,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5582848,"data_stored":43192387,"data_compressed":1416998,"data_compressed_allocated":3502080,"data_compressed_original":42148296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4386816,"data_stored":38728056,"data_compressed":1232532,"data_compressed_allocated":2830336,"data_compressed_original":37929993,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":310283,"data_compressed":48933,"data_compressed_allocated":81920,"data_compressed_original":241664,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2674688,"data_stored":23980310,"data_compressed":785471,"data_compressed_allocated":1888256,"data_compressed_original":23563434,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:00:48.893 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [7, 0] 2024-09-14T07:00:48.893 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.8', '7', '0'] 2024-09-14T07:00:48.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.8 7 0 2024-09-14T07:00:48.895 INFO:teuthology.orchestra.run.smithi007.stdout:5838 trunc from 0xe608653 to 0x7f8d315 2024-09-14T07:00:48.904 INFO:teuthology.orchestra.run.smithi007.stdout:5839 read 0xef75bc thru 0xef8e71 (0x18b6 bytes) 2024-09-14T07:00:48.905 INFO:teuthology.orchestra.run.smithi007.stdout:5840 read 0x2b14df7 thru 0x2b24476 (0xf680 bytes) 2024-09-14T07:00:48.906 INFO:teuthology.orchestra.run.smithi007.stdout:5841 read 0x36ee07b thru 0x36f6f46 (0x8ecc bytes) 2024-09-14T07:00:48.906 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.906 INFO:teuthology.orchestra.run.smithi007.stdout:5845 write 0x27afe9a thru 0x27bb152 (0xb2b9 bytes) 2024-09-14T07:00:48.910 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.910 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.919 INFO:teuthology.orchestra.run.smithi007.stdout:5852 writesame 0xb327fd7 thru 0xb32dec6 data_size 0x3e(0x5ef0 bytes) 2024-09-14T07:00:48.919 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:48.918+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:48.919 INFO:teuthology.orchestra.run.smithi007.stdout:5853 write 0x24cd5f6 thru 0x24db544 (0xdf4f bytes) 2024-09-14T07:00:48.921 INFO:teuthology.orchestra.run.smithi007.stdout:5854 write 0x87ce121 thru 0x87d9e81 (0xbd61 bytes) 2024-09-14T07:00:48.924 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:48.925 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:48.925 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:48.925 INFO:teuthology.orchestra.run.smithi007.stdout:5855 trunc from 0xb32dec7 to 0xbff7b06 2024-09-14T07:00:48.930 INFO:teuthology.orchestra.run.smithi007.stdout:5857 writesame 0x7847511 thru 0x784ffb2 data_size 0x2a(0x8aa2 bytes) 2024-09-14T07:00:48.933 INFO:teuthology.orchestra.run.smithi007.stdout:5858 flatten 2024-09-14T07:00:48.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:48.940 INFO:teuthology.orchestra.run.smithi007.stdout:5859 read 0x86ec879 thru 0x86fae67 (0xe5ef bytes) 2024-09-14T07:00:48.940 INFO:teuthology.orchestra.run.smithi007.stdout:5861 write 0x9994032 thru 0x9995c09 (0x1bd8 bytes) 2024-09-14T07:00:48.945 INFO:teuthology.orchestra.run.smithi007.stdout:5863 write 0x49f402b thru 0x49fca3e (0x8a14 bytes) 2024-09-14T07:00:48.947 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.947 INFO:teuthology.orchestra.run.smithi007.stdout:5865 read 0x85c534f thru 0x85c7434 (0x20e6 bytes) 2024-09-14T07:00:48.947 INFO:teuthology.orchestra.run.smithi007.stdout:5868 punch from 0xa82b43d to 0xa838701, (0xd2c4 bytes) 2024-09-14T07:00:48.948 INFO:teuthology.orchestra.run.smithi007.stdout:5869 read 0x6c1f928 thru 0x6c26887 (0x6f60 bytes) 2024-09-14T07:00:48.948 INFO:teuthology.orchestra.run.smithi007.stdout:5870 write 0xab76cd8 thru 0xab7b1f8 (0x4521 bytes) 2024-09-14T07:00:48.957 INFO:teuthology.orchestra.run.smithi007.stdout:5871 trunc from 0xbff7b06 to 0x72f7445 2024-09-14T07:00:48.972 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.972 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:48.981 INFO:teuthology.orchestra.run.smithi007.stdout:5876 write 0x98054a9 thru 0x980f374 (0x9ecc bytes) 2024-09-14T07:00:48.982 INFO:teuthology.orchestra.run.smithi007.stdout:5877 write 0x4aaa304 thru 0x4ab59df (0xb6dc bytes) 2024-09-14T07:00:48.996 INFO:teuthology.orchestra.run.smithi007.stdout:5878 writesame 0xe5e722f thru 0xe5e8936 data_size 0x16(0x1708 bytes) 2024-09-14T07:00:48.998 INFO:teuthology.orchestra.run.smithi007.stdout:5879 read 0x5f35e09 thru 0x5f3ddd4 (0x7fcc bytes) 2024-09-14T07:00:49.009 INFO:teuthology.orchestra.run.smithi007.stdout:5880 trunc from 0xe5e8937 to 0x525a399 2024-09-14T07:00:49.019 INFO:teuthology.orchestra.run.smithi007.stdout:5881 clone 33 order 18 su 65536 sc 3 2024-09-14T07:00:49.020 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:49.019+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:49.067 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:49.067 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.311443792999995, 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:49.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:00:49.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:00:49.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:00:49.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.713031201000007, 2024-09-14T07:00:49.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:00:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:00:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:49.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:00:49.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:00:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:49.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:00:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:00:49.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:00:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:00:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:00:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:00:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:00:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:00:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:00:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:00:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:00:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:00:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:00:49.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:00:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:00:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:00:49.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:00:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:00:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:00:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:00:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:00:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:00:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:00:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:00:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:00:49.091 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.707371558000006, 2024-09-14T07:00:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:00:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:00:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:00:49.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:00:49.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:00:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:00:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:00:49.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:00:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:00:49.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:00:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:00:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:00:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:00:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:00:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:00:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:00:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:00:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:00:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:00:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:00:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:00:49.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:00:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:00:49.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:00:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:00:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:00:49.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:00:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:00:49.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:00:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:00:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.110 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:49.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.401535279999997, 2024-09-14T07:00:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:00:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:00:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:00:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:00:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:00:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:00:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:00:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:00:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:00:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:00:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:00:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:00:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:00:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:00:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:00:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:00:49.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:00:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.401504488, 2024-09-14T07:00:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:00:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:00:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:49.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:00:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:00:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:00:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:00:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:00:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:00:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:00:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:00:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:00:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:00:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:00:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:00:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:00:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.401481118, 2024-09-14T07:00:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:00:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:00:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:00:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:00:49.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:00:49.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:00:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:00:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:00:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:00:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:00:49.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:00:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:00:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:00:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:00:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:00:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:00:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:49.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.400674608999999, 2024-09-14T07:00:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:00:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:00:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:00:49.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:49.153 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.400652215999997, 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:00:49.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.400632076000001, 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:00:49.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:49.162 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.400481921000001, 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:49.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:00:49.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.400426367000001, 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:00:49.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:00:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.400393326, 2024-09-14T07:00:49.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.398040240999997, 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.398017676000002, 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.184 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.397160681000003, 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.39594366, 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:00:49.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.395815820000003, 2024-09-14T07:00:49.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:00:49.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.395795726999999, 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:00:49.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.394768833999997, 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:00:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138497 3.14 3:298b17d3:::rbd_data.114f74bd3bbf.00000000000002d7:head [set-alloc-hint object_size 262144 write_size 262144,write 218355~43789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.747261552000001, 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97686345699999999, 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138497 2024-09-14T07:00:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317983+0000", 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318003+0000", 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0610000000000001e-05 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318022+0000", 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8627e-05 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318030+0000", 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318101+0000", 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1162999999999996e-05 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318108+0000", 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-14T07:00:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291257+0000", 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97314914100000005 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291396+0000", 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013883799999999999 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291573+0000", 2024-09-14T07:00:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017716800000000001 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.293147+0000", 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015740400000000001 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294753+0000", 2024-09-14T07:00:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00160597 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294792+0000", 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8758999999999999e-05 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294844+0000", 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2188000000000002e-05 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:49.214 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:49.215 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:49.216 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:49.214+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:49.316 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:49.315+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:49.319 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:49.319 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:49.319 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:49.319 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:49.327 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:49.417 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:49.416+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:49.431 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:49.431 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:49.431 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:49.431 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:49.431 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:49.439 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:49.522 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:49.522+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:49.582 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.8 pg_upmap mapping to [7,0] 2024-09-14T07:00:49.587 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:49.587 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:49.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:49.587 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:49.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.588 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131358 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.588 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:49.588 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.086023435000001, 2024-09-14T07:00:49.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030805583, 2024-09-14T07:00:49.588 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.588 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.589 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.589 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.589 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.589 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.589 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131358 2024-09-14T07:00:49.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.590 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:49.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:49.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:00:49.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:00:49.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498000+0000", 2024-09-14T07:00:49.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:00:49.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498004+0000", 2024-09-14T07:00:49.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2569999999999999e-06 2024-09-14T07:00:49.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498007+0000", 2024-09-14T07:00:49.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.507e-06 2024-09-14T07:00:49.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499958+0000", 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001950989 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499962+0000", 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528792+0000", 2024-09-14T07:00:49.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288306389999999 2024-09-14T07:00:49.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528804+0000", 2024-09-14T07:00:49.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2096000000000001e-05 2024-09-14T07:00:49.596 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.596 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131359 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.086004025000001, 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308363920000001, 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.597 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131359 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:49.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:00:49.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498019+0000", 2024-09-14T07:00:49.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:00:49.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498020+0000", 2024-09-14T07:00:49.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4529999999999999e-06 2024-09-14T07:00:49.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498023+0000", 2024-09-14T07:00:49.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3450000000000002e-06 2024-09-14T07:00:49.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498026+0000", 2024-09-14T07:00:49.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.661e-06 2024-09-14T07:00:49.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499995+0000", 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00196926 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500002+0000", 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6930000000000001e-06 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528842+0000", 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288401300000001 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528855+0000", 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2595999999999999e-05 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.603 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.604 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.604 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131360 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.604 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:49.604 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085993021999997, 2024-09-14T07:00:49.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030841943, 2024-09-14T07:00:49.604 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.605 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.605 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.605 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.605 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.605 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.606 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131360 2024-09-14T07:00:49.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.606 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:49.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:00:49.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498030+0000", 2024-09-14T07:00:49.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:49.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498031+0000", 2024-09-14T07:00:49.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1119999999999999e-06 2024-09-14T07:00:49.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498034+0000", 2024-09-14T07:00:49.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:00:49.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498036+0000", 2024-09-14T07:00:49.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.599e-06 2024-09-14T07:00:49.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500030+0000", 2024-09-14T07:00:49.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001993613 2024-09-14T07:00:49.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500034+0000", 2024-09-14T07:00:49.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1060000000000002e-06 2024-09-14T07:00:49.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528867+0000", 2024-09-14T07:00:49.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288327429999999 2024-09-14T07:00:49.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528871+0000", 2024-09-14T07:00:49.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.673e-06 2024-09-14T07:00:49.615 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.615 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131361 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085972277000003, 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308644520000001, 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.616 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131361 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498052+0000", 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.24e-06 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498055+0000", 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1609999999999999e-06 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498058+0000", 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.05e-06 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500066+0000", 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020081169999999998 2024-09-14T07:00:49.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500075+0000", 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0559999999999994e-06 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528903+0000", 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288276810000001 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528915+0000", 2024-09-14T07:00:49.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1938e-05 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131362 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085953496000002, 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030860661, 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.621 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131362 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498075+0000", 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5639999999999996e-06 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498079+0000", 2024-09-14T07:00:49.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498081+0000", 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8660000000000002e-06 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500101+0000", 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020191219999999999 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500104+0000", 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6890000000000002e-06 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528925+0000", 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028820971 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528930+0000", 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3390000000000003e-06 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131363 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085937248, 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030874107, 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131363 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.626 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:49.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498086+0000", 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1009999999999999e-06 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498090+0000", 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5609999999999999e-06 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498093+0000", 2024-09-14T07:00:49.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7360000000000001e-06 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500136+0000", 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020436030000000002 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500145+0000", 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9530000000000005e-06 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528948+0000", 2024-09-14T07:00:49.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028803103 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528959+0000", 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0808999999999999e-05 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131364 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:49.630 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085920866999999, 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308734129999999, 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131364 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.631 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498102+0000", 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:00:49.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498103+0000", 2024-09-14T07:00:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498107+0000", 2024-09-14T07:00:49.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4089999999999999e-06 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498110+0000", 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7760000000000002e-06 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500177+0000", 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020678200000000002 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500183+0000", 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4979999999999997e-06 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528970+0000", 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287867049999999 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528975+0000", 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1959999999999999e-06 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131365 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085898376000003, 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886024, 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131365 2024-09-14T07:00:49.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498125+0000", 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.17e-06 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498129+0000", 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498131+0000", 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.666e-06 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500210+0000", 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020789559999999999 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500215+0000", 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:00:49.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528993+0000", 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028777174 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529010+0000", 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7419999999999999e-05 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131366 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.640 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085880078000002, 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308831409999999, 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131366 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:49.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498143+0000", 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498144+0000", 2024-09-14T07:00:49.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.474e-06 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498147+0000", 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.371e-06 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498150+0000", 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6410000000000002e-06 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500239+0000", 2024-09-14T07:00:49.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002088842 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500243+0000", 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6689999999999999e-06 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529020+0000", 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287776420000001 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529025+0000", 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2440000000000001e-06 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131367 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085869017999997, 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030895135, 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.645 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131367 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498154+0000", 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2100000000000001e-07 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498155+0000", 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1310000000000001e-06 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498157+0000", 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7870000000000002e-06 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498160+0000", 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9009999999999998e-06 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500264+0000", 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021031489999999999 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500268+0000", 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0929999999999996e-06 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529043+0000", 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287752139999999 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529048+0000", 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6389999999999997e-06 2024-09-14T07:00:49.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131368 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085844885999997, 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308865389999999, 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.650 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131368 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498178+0000", 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8200000000000001e-07 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498179+0000", 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6929999999999999e-06 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498183+0000", 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9439999999999998e-06 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498188+0000", 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9409999999999998e-06 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500287+0000", 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020982269999999998 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500290+0000", 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6349999999999999e-06 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529059+0000", 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028768648 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529064+0000", 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.169e-06 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.654 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131369 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085829693999997, 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886017, 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131369 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:49.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.28e-07 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498194+0000", 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1459999999999999e-06 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498197+0000", 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0620000000000001e-06 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498200+0000", 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6000000000000001e-06 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500310+0000", 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021101840000000002 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500314+0000", 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.126e-06 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529074+0000", 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.02875961 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529079+0000", 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0610000000000003e-06 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131370 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085813901000002, 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308930119999999, 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.659 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131370 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498209+0000", 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498210+0000", 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.187e-06 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498213+0000", 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.151e-06 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498216+0000", 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8289999999999999e-06 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500340+0000", 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021244229999999998 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500349+0000", 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529096+0000", 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287473309999999 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529101+0000", 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.663 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131371 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085795521000001, 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308901880000001, 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.664 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131371 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498229+0000", 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1110000000000002e-06 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498234+0000", 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498239+0000", 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.651e-06 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500382+0000", 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021424780000000002 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500389+0000", 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529112+0000", 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287229410000001 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529117+0000", 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9520000000000002e-06 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131372 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085774405000002, 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030884366, 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131372 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8599999999999999e-07 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498251+0000", 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498257+0000", 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6370000000000002e-06 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498262+0000", 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500418+0000", 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021561800000000002 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500423+0000", 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1520000000000002e-06 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529127+0000", 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287049150000001 2024-09-14T07:00:49.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529132+0000", 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.865e-06 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131373 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085741341999999, 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308673479999999, 2024-09-14T07:00:49.673 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131373 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498283+0000", 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498290+0000", 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498295+0000", 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3759999999999999e-06 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500441+0000", 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021462849999999999 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500445+0000", 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8310000000000003e-06 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529142+0000", 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028697363 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529148+0000", 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.677 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131374 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085707726000003, 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308477090000001, 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.678 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131374 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.679 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498322+0000", 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5760000000000002e-06 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498328+0000", 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4609999999999996e-06 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498334+0000", 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7130000000000002e-06 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500464+0000", 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002130617 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500468+0000", 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7340000000000002e-06 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529157+0000", 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286893669999999 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529162+0000", 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131375 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:49.682 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085667589000003, 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308296320000001, 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131375 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498358+0000", 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.317e-06 2024-09-14T07:00:49.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498364+0000", 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5930000000000002e-06 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498369+0000", 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3630000000000002e-06 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500488+0000", 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021187699999999999 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500492+0000", 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529179+0000", 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286873590000001 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529184+0000", 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131376 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085640050000002, 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030817713, 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.687 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131376 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498383+0000", 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498385+0000", 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.091e-06 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498391+0000", 2024-09-14T07:00:49.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1070000000000001e-06 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498396+0000", 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4670000000000003e-06 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500514+0000", 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021177449999999999 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500518+0000", 2024-09-14T07:00:49.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8770000000000003e-06 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529195+0000", 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028676938 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529200+0000", 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131377 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 33.085600524999997, 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308008749999999, 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131377 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:49.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:00:49.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498425+0000", 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5620000000000002e-06 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498431+0000", 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2539999999999998e-06 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498436+0000", 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.31e-06 2024-09-14T07:00:49.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500539+0000", 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021025459999999998 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500543+0000", 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.597e-06 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529218+0000", 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286753930000001 2024-09-14T07:00:49.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529223+0000", 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.758e-06 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:49.696 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:49.698 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:49.699 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:49.697+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:49.717 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-14T07:00:49.799 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:49.798+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:49.882 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:49.882 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:49.883 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:49.883 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:49.892 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:49.936 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:49.936+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:50.001 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:50.000+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:50.036 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:50.037 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:50.037 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:50.047 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:50.131 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:50.130+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:50.200 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:50.200 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:50.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:50.200 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:50.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.033377990999995, 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:00:50.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:00:50.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.382034886, 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:00:50.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.382019417999999, 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:00:50.210 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:00:50.214 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.382008116999998, 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.215 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:00:50.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.218 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.219 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375264643999998, 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:50.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:00:50.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:00:50.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375252039999999, 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.224 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:50.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.226 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:00:50.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.229 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.229 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:50.229 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375232523999998, 2024-09-14T07:00:50.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:50.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:00:50.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375220483, 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.234 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:00:50.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375200180999997, 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:00:50.238 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375187865000001, 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.243 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375176424999999, 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.247 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:50.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:00:50.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.251 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375165926000001, 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.252 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:00:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:00:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:00:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:00:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375155733, 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.256 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:00:50.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:00:50.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.260 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375140186000003, 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375110577999997, 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.268 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375088927999997, 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:00:50.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:00:50.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.375066689000001, 2024-09-14T07:00:50.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:00:50.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.277 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.369470323999998, 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.278 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:50.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:00:50.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.369460271000001, 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.282 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:00:50.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:118426 3.b 3:d50c2793:::rbd_data.114f74d20483.000000000000001c:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 37.698078467999999, 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.95135023100000005, 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 118426 2024-09-14T07:00:50.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500745+0000", 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500748+0000", 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1329999999999999e-06 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500759+0000", 2024-09-14T07:00:50.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0475000000000001e-05 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500765+0000", 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500824+0000", 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:50.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500830+0000", 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5889999999999999e-06 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450787+0000", 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.94995644099999998 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450901+0000", 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114032 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451052+0000", 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015113 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451746+0000", 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069442799999999997 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452006+0000", 2024-09-14T07:00:50.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025975 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452059+0000", 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3090000000000002e-05 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452095+0000", 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5395999999999998e-05 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.294 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.294 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:50.295 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:50.296 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:50.294+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:50.396 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:50.395+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:50.450 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:50.450 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:50.450 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:50.450 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:50.459 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:50.500 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:50.499+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:50.597 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:50.597+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:50.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:00:50.612 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:50.612 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:50.612 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:50.612 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:50.612 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:50.623 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:50.698 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:50.698+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:50.770 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:00:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 90.005131883999994, 2024-09-14T07:00:50.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:00:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:00:50.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.932012618000002, 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:00:50.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:00:50.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:00:50.782 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.931755964999994, 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.783 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:00:50.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:00:50.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.905468890999998, 2024-09-14T07:00:50.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.791 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.905344276999998, 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:50.792 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:50.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:00:50.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:00:50.796 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.410725115000005, 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:00:50.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.136322454999998, 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:50.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:00:50.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:00:50.805 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 67.141706845000002, 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:00:50.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.951534378999995, 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:00:50.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:00:50.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:50.814 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.902401088999994, 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:50.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:00:50.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.534182125000001, 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:00:50.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:00:50.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:50.823 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.413613186000006, 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:50.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:00:50.827 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.282697304999999, 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:00:50.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.832 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.253660042999996, 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:00:50.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:00:50.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.227890621, 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.837 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:00:50.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:00:50.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:50.841 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.194532477999999, 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:50.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.845 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.030324696999998, 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:00:50.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.018945436999999, 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.850 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:00:50.854 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.018897742999997, 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:00:50.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:00:50.858 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 66.018555418000005, 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:50.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:00:50.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:50.863 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:50.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:50.865 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:50.866 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:50.864+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:50.965 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:50.965+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:50.987 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:50.988 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:51.004 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:51.005 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:51.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:51.021 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:51.032 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:51.032 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:51.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.032 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172371 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.032 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851073+0000", 2024-09-14T07:00:51.032 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161689354, 2024-09-14T07:00:51.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.161707136, 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172371 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851073+0000", 2024-09-14T07:00:51.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851073+0000", 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851074+0000", 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851075+0000", 2024-09-14T07:00:51.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1340000000000001e-06 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851078+0000", 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0860000000000002e-06 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851080+0000", 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5059999999999998e-06 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851330+0000", 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00024954399999999998 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851337+0000", 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5449999999999998e-06 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596781+0000", 2024-09-14T07:00:51.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74544407700000004 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596789+0000", 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0800000000000006e-06 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172814 3.8 3.9c667668 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1612106680000001, 2024-09-14T07:00:51.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1613028569999999, 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172814 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:00:51.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592075+0000", 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74052290899999995 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592081+0000", 2024-09-14T07:00:51.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6339999999999998e-06 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592090+0000", 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5929999999999995e-06 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592097+0000", 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4930000000000003e-06 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597854+0000", 2024-09-14T07:00:51.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0057574890000000002 2024-09-14T07:00:51.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.041 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.041 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597858+0000", 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.228e-06 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173956 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592577+0000", 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42018605199999998, 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42030842099999999, 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.042 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173956 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592577+0000", 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592577+0000", 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592577+0000", 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4699999999999998e-07 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592578+0000", 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7699999999999992e-07 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592581+0000", 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592584+0000", 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7159999999999999e-06 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598976+0000", 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0063924070000000001 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598980+0000", 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6239999999999999e-06 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174914 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593404+0000", 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41935872699999999, 2024-09-14T07:00:51.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41950952200000002, 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174914 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593404+0000", 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593404+0000", 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593405+0000", 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9500000000000002e-07 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593407+0000", 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0059999999999999e-06 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593413+0000", 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.037e-06 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593417+0000", 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8420000000000004e-06 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600076+0000", 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066581019999999999 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600079+0000", 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6509999999999999e-06 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175557 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594295+0000", 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41846773300000001, 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41864697099999998, 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175557 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594295+0000", 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594295+0000", 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594295+0000", 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:00:51.052 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594297+0000", 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.069e-06 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594304+0000", 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3239999999999999e-06 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594315+0000", 2024-09-14T07:00:51.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.133e-05 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601170+0000", 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068549500000000003 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601174+0000", 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0389999999999998e-06 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:178882 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.609209+0000", 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.403554044, 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40376197000000003, 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 178882 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609209+0000", 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609209+0000", 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.056 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609209+0000", 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609211+0000", 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.844e-06 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609217+0000", 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8699999999999997e-06 2024-09-14T07:00:51.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.058 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609221+0000", 2024-09-14T07:00:51.058 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4900000000000002e-06 2024-09-14T07:00:51.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.058 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609359+0000", 2024-09-14T07:00:51.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013754699999999999 2024-09-14T07:00:51.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609368+0000", 2024-09-14T07:00:51.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9050000000000003e-06 2024-09-14T07:00:51.059 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.060 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.060 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.060 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.060 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179908 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.060 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615147+0000", 2024-09-14T07:00:51.061 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39761590600000002, 2024-09-14T07:00:51.061 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39785169300000001, 2024-09-14T07:00:51.061 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.061 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.061 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.061 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.062 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.062 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.062 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179908 2024-09-14T07:00:51.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.062 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.062 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.062 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615147+0000", 2024-09-14T07:00:51.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615147+0000", 2024-09-14T07:00:51.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615147+0000", 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.39e-07 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615153+0000", 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1999999999999999e-06 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615159+0000", 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0419999999999999e-06 2024-09-14T07:00:51.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615164+0000", 2024-09-14T07:00:51.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3420000000000001e-06 2024-09-14T07:00:51.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615425+0000", 2024-09-14T07:00:51.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026182100000000003 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615434+0000", 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0430000000000008e-06 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180592 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.066 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.620968+0000", 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39179499000000001, 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39205946600000002, 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180592 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620968+0000", 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620968+0000", 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620968+0000", 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.67e-07 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620969+0000", 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4139999999999999e-06 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620973+0000", 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7689999999999998e-06 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620976+0000", 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9270000000000001e-06 2024-09-14T07:00:51.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621194+0000", 2024-09-14T07:00:51.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000217846 2024-09-14T07:00:51.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.071 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621201+0000", 2024-09-14T07:00:51.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4229999999999999e-06 2024-09-14T07:00:51.071 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.071 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.071 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.072 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180767 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.072 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621570+0000", 2024-09-14T07:00:51.072 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39119272500000002, 2024-09-14T07:00:51.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39148503200000001, 2024-09-14T07:00:51.072 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.073 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.073 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.073 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.073 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.073 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.073 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180767 2024-09-14T07:00:51.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.073 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621570+0000", 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621570+0000", 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621570+0000", 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.15e-07 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621572+0000", 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.748e-06 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621578+0000", 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7200000000000003e-06 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621582+0000", 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6079999999999998e-06 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.622068+0000", 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048608599999999999 2024-09-14T07:00:51.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.622072+0000", 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6550000000000002e-06 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172372 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851084+0000", 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1616791399999999, 2024-09-14T07:00:51.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.162016492, 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172372 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851084+0000", 2024-09-14T07:00:51.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851084+0000", 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851084+0000", 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851085+0000", 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.35e-06 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851088+0000", 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.021e-06 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851091+0000", 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.407e-06 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.080 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851365+0000", 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00027409500000000002 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851368+0000", 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5829999999999998e-06 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596819+0000", 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745450744 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596826+0000", 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4229999999999999e-06 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172833 3.8 3.e779d28 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592110+0000", 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42065275800000002, 2024-09-14T07:00:51.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42102161199999999, 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172833 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592110+0000", 2024-09-14T07:00:51.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592110+0000", 2024-09-14T07:00:51.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592110+0000", 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0999999999999999e-07 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592112+0000", 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.73e-06 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592116+0000", 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1999999999999996e-06 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592119+0000", 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1939999999999998e-06 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597897+0000", 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0057770740000000001 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597904+0000", 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7999999999999999e-06 2024-09-14T07:00:51.086 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173958 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592587+0000", 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.420175558, 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.420571372, 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.087 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173958 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592587+0000", 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592587+0000", 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592587+0000", 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592589+0000", 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.089 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592592+0000", 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592594+0000", 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5919999999999999e-06 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599009+0000", 2024-09-14T07:00:51.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0064148110000000003 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599017+0000", 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8509999999999999e-06 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174916 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593437+0000", 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41932554799999999, 2024-09-14T07:00:51.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41974819699999999, 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174916 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593437+0000", 2024-09-14T07:00:51.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593437+0000", 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593438+0000", 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1500000000000004e-07 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593440+0000", 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9700000000000002e-06 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593446+0000", 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.161e-06 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593451+0000", 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6990000000000002e-06 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600113+0000", 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066623300000000002 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600120+0000", 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5900000000000002e-06 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175652 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594323+0000", 2024-09-14T07:00:51.095 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41843992800000002, 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41888856400000002, 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175652 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594323+0000", 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594323+0000", 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594323+0000", 2024-09-14T07:00:51.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0399999999999996e-07 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594325+0000", 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9580000000000001e-06 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594331+0000", 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8490000000000002e-06 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594336+0000", 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8520000000000003e-06 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601203+0000", 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068674729999999998 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601211+0000", 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7859999999999997e-06 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.099 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:178884 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.609228+0000", 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40353486100000002, 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40401264199999998, 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 178884 2024-09-14T07:00:51.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609228+0000", 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609228+0000", 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609228+0000", 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2599999999999998e-07 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609230+0000", 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7710000000000001e-06 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609236+0000", 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4420000000000003e-06 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609241+0000", 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6e-06 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609403+0000", 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000162017 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609412+0000", 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6249999999999996e-06 2024-09-14T07:00:51.103 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179909 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615170+0000", 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.397593007, 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39809724200000002, 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.104 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179909 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615170+0000", 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615170+0000", 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615170+0000", 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615172+0000", 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6190000000000001e-06 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615182+0000", 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0319000000000001e-05 2024-09-14T07:00:51.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615186+0000", 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2950000000000004e-06 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615469+0000", 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028310100000000001 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615477+0000", 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6490000000000008e-06 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180769 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621593+0000", 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39116931500000002, 2024-09-14T07:00:51.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39169966499999997, 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180769 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621593+0000", 2024-09-14T07:00:51.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621593+0000", 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621594+0000", 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6199999999999998e-07 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621595+0000", 2024-09-14T07:00:51.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:00:51.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621599+0000", 2024-09-14T07:00:51.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9759999999999997e-06 2024-09-14T07:00:51.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621603+0000", 2024-09-14T07:00:51.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6370000000000001e-06 2024-09-14T07:00:51.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.622085+0000", 2024-09-14T07:00:51.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048180199999999998 2024-09-14T07:00:51.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.113 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.622089+0000", 2024-09-14T07:00:51.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1960000000000001e-06 2024-09-14T07:00:51.113 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.113 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.114 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.114 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172373 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.114 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851094+0000", 2024-09-14T07:00:51.114 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161669093, 2024-09-14T07:00:51.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1622255290000001, 2024-09-14T07:00:51.114 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.115 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.115 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.115 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.115 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.115 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.115 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172373 2024-09-14T07:00:51.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.115 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851094+0000", 2024-09-14T07:00:51.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851094+0000", 2024-09-14T07:00:51.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851094+0000", 2024-09-14T07:00:51.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:51.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851095+0000", 2024-09-14T07:00:51.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T07:00:51.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851098+0000", 2024-09-14T07:00:51.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T07:00:51.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851101+0000", 2024-09-14T07:00:51.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5859999999999999e-06 2024-09-14T07:00:51.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851400+0000", 2024-09-14T07:00:51.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029982099999999997 2024-09-14T07:00:51.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851408+0000", 2024-09-14T07:00:51.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7789999999999996e-06 2024-09-14T07:00:51.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596860+0000", 2024-09-14T07:00:51.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74545224899999996 2024-09-14T07:00:51.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596868+0000", 2024-09-14T07:00:51.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.661e-06 2024-09-14T07:00:51.123 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.124 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.124 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.124 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172901 3.8 3.cb984c08 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.124 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592125+0000", 2024-09-14T07:00:51.125 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42063787400000002, 2024-09-14T07:00:51.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.421232094, 2024-09-14T07:00:51.125 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.125 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.125 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.125 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.126 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.126 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.126 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172901 2024-09-14T07:00:51.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.126 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592125+0000", 2024-09-14T07:00:51.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592125+0000", 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592125+0000", 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592127+0000", 2024-09-14T07:00:51.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.689e-06 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592131+0000", 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6660000000000001e-06 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592133+0000", 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6699999999999998e-06 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597931+0000", 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.005797399 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597937+0000", 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6030000000000001e-06 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173959 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.130 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592610+0000", 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42015312700000002, 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42077459499999997, 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173959 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592610+0000", 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592610+0000", 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592610+0000", 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6100000000000001e-07 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592621+0000", 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0713999999999999e-05 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592625+0000", 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1829999999999996e-06 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592628+0000", 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8700000000000001e-06 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599040+0000", 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0064125170000000004 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599044+0000", 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7579999999999998e-06 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174918 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593457+0000", 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41930593199999999, 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41995389799999999, 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174918 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.135 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593457+0000", 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593457+0000", 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593457+0000", 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593459+0000", 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593466+0000", 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9070000000000001e-06 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.137 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.138 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.138 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593471+0000", 2024-09-14T07:00:51.138 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3229999999999997e-06 2024-09-14T07:00:51.138 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.138 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.139 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600145+0000", 2024-09-14T07:00:51.139 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006674246 2024-09-14T07:00:51.139 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.139 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.139 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.140 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600152+0000", 2024-09-14T07:00:51.140 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0439999999999998e-06 2024-09-14T07:00:51.140 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.140 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.140 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.140 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.141 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.141 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175653 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.141 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594342+0000", 2024-09-14T07:00:51.141 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41842032600000001, 2024-09-14T07:00:51.141 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41909426500000002, 2024-09-14T07:00:51.142 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.144 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.144 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.145 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.145 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.145 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.145 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175653 2024-09-14T07:00:51.145 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.145 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.146 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.146 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594342+0000", 2024-09-14T07:00:51.146 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.146 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.146 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.147 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.147 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594342+0000", 2024-09-14T07:00:51.147 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.147 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.147 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594343+0000", 2024-09-14T07:00:51.148 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6199999999999999e-07 2024-09-14T07:00:51.148 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.148 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.148 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.148 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594345+0000", 2024-09-14T07:00:51.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8870000000000001e-06 2024-09-14T07:00:51.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.149 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.149 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.149 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594351+0000", 2024-09-14T07:00:51.149 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3010000000000003e-06 2024-09-14T07:00:51.149 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.150 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.150 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594355+0000", 2024-09-14T07:00:51.150 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.019e-06 2024-09-14T07:00:51.150 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.150 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601236+0000", 2024-09-14T07:00:51.151 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006881435 2024-09-14T07:00:51.151 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.151 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.151 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.151 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601240+0000", 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6289999999999999e-06 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:178886 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.609259+0000", 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40350397100000002, 2024-09-14T07:00:51.152 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40420598600000002, 2024-09-14T07:00:51.153 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.153 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.153 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.153 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.153 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.153 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.153 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 178886 2024-09-14T07:00:51.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.154 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.154 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.154 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609259+0000", 2024-09-14T07:00:51.154 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.154 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.154 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609259+0000", 2024-09-14T07:00:51.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.155 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.155 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.155 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609259+0000", 2024-09-14T07:00:51.155 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5400000000000002e-07 2024-09-14T07:00:51.155 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.156 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609261+0000", 2024-09-14T07:00:51.156 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6220000000000001e-06 2024-09-14T07:00:51.156 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.156 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.156 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609265+0000", 2024-09-14T07:00:51.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3429999999999998e-06 2024-09-14T07:00:51.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.157 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.157 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.157 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609268+0000", 2024-09-14T07:00:51.157 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1949999999999999e-06 2024-09-14T07:00:51.157 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.158 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.158 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609448+0000", 2024-09-14T07:00:51.158 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017975000000000001 2024-09-14T07:00:51.158 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.158 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.159 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.159 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609456+0000", 2024-09-14T07:00:51.159 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1759999999999993e-06 2024-09-14T07:00:51.159 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.159 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.159 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.159 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.160 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.160 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179910 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.160 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615208+0000", 2024-09-14T07:00:51.160 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39755478599999999, 2024-09-14T07:00:51.160 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39828316000000002, 2024-09-14T07:00:51.160 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.161 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.161 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.161 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.161 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.161 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.161 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179910 2024-09-14T07:00:51.161 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.162 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.162 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615208+0000", 2024-09-14T07:00:51.162 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.162 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.162 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.162 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615208+0000", 2024-09-14T07:00:51.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.163 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.163 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.163 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.163 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615208+0000", 2024-09-14T07:00:51.163 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1199999999999998e-07 2024-09-14T07:00:51.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.164 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.164 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.164 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615210+0000", 2024-09-14T07:00:51.164 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2729999999999999e-06 2024-09-14T07:00:51.164 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615214+0000", 2024-09-14T07:00:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0280000000000002e-06 2024-09-14T07:00:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.165 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615216+0000", 2024-09-14T07:00:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.728e-06 2024-09-14T07:00:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.166 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615511+0000", 2024-09-14T07:00:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029470199999999998 2024-09-14T07:00:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.167 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615519+0000", 2024-09-14T07:00:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5610000000000001e-06 2024-09-14T07:00:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.168 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180771 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621606+0000", 2024-09-14T07:00:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39115621299999997, 2024-09-14T07:00:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39191208599999999, 2024-09-14T07:00:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.169 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180771 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.170 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621606+0000", 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621606+0000", 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627711+0000", 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00610459 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.171 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627722+0000", 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.095e-05 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627748+0000", 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6188999999999999e-05 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627762+0000", 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4117000000000001e-05 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.172 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627876+0000", 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113225 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627905+0000", 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9255e-05 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172374 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.173 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851110+0000", 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1616529170000001, 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1624363719999999, 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172374 2024-09-14T07:00:51.174 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851110+0000", 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851110+0000", 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.175 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851110+0000", 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.53e-07 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851111+0000", 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.446e-06 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.176 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851115+0000", 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851118+0000", 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0180000000000002e-06 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851432+0000", 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00031424600000000001 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.177 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851436+0000", 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596914+0000", 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74547831099999995 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596919+0000", 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1800000000000004e-06 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.178 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172906 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592137+0000", 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42062546499999998, 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42144072799999999, 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.179 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172906 2024-09-14T07:00:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592137+0000", 2024-09-14T07:00:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.180 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592137+0000", 2024-09-14T07:00:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.181 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592137+0000", 2024-09-14T07:00:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592139+0000", 2024-09-14T07:00:51.182 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.226e-06 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592142+0000", 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3210000000000001e-06 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592145+0000", 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.083e-06 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.183 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597969+0000", 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0058238109999999999 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597976+0000", 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4889999999999998e-06 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.184 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173961 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592632+0000", 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42013066900000001, 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42099743299999998, 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173961 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.185 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592632+0000", 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592632+0000", 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592632+0000", 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9099999999999999e-07 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.186 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592634+0000", 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3540000000000001e-06 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592637+0000", 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592640+0000", 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6390000000000001e-06 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.187 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599073+0000", 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0064334320000000002 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599081+0000", 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6040000000000004e-06 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.188 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175226 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593477+0000", 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.419285772, 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42017915300000003, 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175226 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.189 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593477+0000", 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593477+0000", 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593477+0000", 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.190 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593479+0000", 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8059999999999999e-06 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593486+0000", 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0659999999999998e-06 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593498+0000", 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1984000000000001e-05 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.191 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600182+0000", 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066840449999999996 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600190+0000", 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6720000000000004e-06 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.192 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175654 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594361+0000", 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41840211100000002, 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41932319600000001, 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175654 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.193 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594361+0000", 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594361+0000", 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594361+0000", 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.194 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594363+0000", 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.863e-06 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594370+0000", 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9539999999999998e-06 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594373+0000", 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6950000000000002e-06 2024-09-14T07:00:51.195 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601269+0000", 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068960710000000001 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601277+0000", 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4379999999999998e-06 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.196 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:178888 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.609272+0000", 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40349069799999998, 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40443827199999999, 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 178888 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.197 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609272+0000", 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609272+0000", 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609272+0000", 2024-09-14T07:00:51.198 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2600000000000001e-07 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609273+0000", 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7999999999999993e-07 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609276+0000", 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2990000000000001e-06 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.199 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609279+0000", 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.328e-06 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609490+0000", 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021163 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609499+0000", 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5169999999999999e-06 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.200 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180773 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.627813+0000", 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38494965799999997, 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38592490400000001, 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.201 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180773 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627813+0000", 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627813+0000", 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.202 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627814+0000", 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.443e-06 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627821+0000", 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6839999999999999e-06 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627836+0000", 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4783000000000001e-05 2024-09-14T07:00:51.203 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627851+0000", 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4962e-05 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627983+0000", 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000132115 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628009+0000", 2024-09-14T07:00:51.204 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5576e-05 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172375 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851122+0000", 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.16164087, 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1626430089999999, 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.205 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172375 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851122+0000", 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851122+0000", 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.206 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851122+0000", 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851125+0000", 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3850000000000002e-06 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851129+0000", 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3450000000000002e-06 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.207 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851131+0000", 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851467+0000", 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00033589399999999999 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851475+0000", 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5429999999999996e-06 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.208 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596955+0000", 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74547975499999997 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596962+0000", 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9820000000000008e-06 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.209 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172908 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592162+0000", 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42060023800000002, 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42163792700000002, 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172908 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.210 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592162+0000", 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592162+0000", 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592163+0000", 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7399999999999998e-07 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.211 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592165+0000", 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6840000000000001e-06 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592169+0000", 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6659999999999999e-06 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592172+0000", 2024-09-14T07:00:51.212 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.785e-06 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598003+0000", 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0058305450000000003 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598009+0000", 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1310000000000003e-06 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.213 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173963 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592650+0000", 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42011284399999999, 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42117851299999998, 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173963 2024-09-14T07:00:51.214 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592650+0000", 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592650+0000", 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.215 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592650+0000", 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5600000000000001e-07 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592652+0000", 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4759999999999999e-06 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592656+0000", 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9389999999999999e-06 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.216 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592658+0000", 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6690000000000002e-06 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599108+0000", 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0064495519999999999 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599112+0000", 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7589999999999999e-06 2024-09-14T07:00:51.217 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175227 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593513+0000", 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41925002, 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42034179799999999, 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.218 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175227 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593513+0000", 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593513+0000", 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.219 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593513+0000", 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8699999999999995e-07 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593515+0000", 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1019999999999999e-06 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593521+0000", 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0780000000000001e-06 2024-09-14T07:00:51.220 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593526+0000", 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7219999999999999e-06 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600214+0000", 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066882720000000003 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600218+0000", 2024-09-14T07:00:51.221 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6650000000000001e-06 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175655 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594378+0000", 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41838494799999998, 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41950426800000001, 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.222 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175655 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594378+0000", 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594378+0000", 2024-09-14T07:00:51.223 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594378+0000", 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.58e-07 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594380+0000", 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.646e-06 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594384+0000", 2024-09-14T07:00:51.224 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594388+0000", 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.281e-06 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601305+0000", 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00691706 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.225 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601309+0000", 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8349999999999997e-06 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:178890 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.609294+0000", 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40346914699999997, 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40461489299999998, 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.226 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 178890 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609294+0000", 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.227 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609294+0000", 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609294+0000", 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5199999999999998e-07 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609303+0000", 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.431e-06 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.228 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609309+0000", 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1229999999999997e-06 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609317+0000", 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9740000000000002e-06 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609523+0000", 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020572599999999999 2024-09-14T07:00:51.229 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609531+0000", 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7819999999999994e-06 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180665 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621027+0000", 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39173524399999998, 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39290892100000002, 2024-09-14T07:00:51.230 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180665 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621027+0000", 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.231 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621027+0000", 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621028+0000", 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6300000000000001e-07 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621035+0000", 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5680000000000002e-06 2024-09-14T07:00:51.232 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621041+0000", 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6999999999999996e-06 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621045+0000", 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.473e-06 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621236+0000", 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0001919 2024-09-14T07:00:51.233 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621244+0000", 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5739999999999998e-06 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180775 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.627876+0000", 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38488677599999999, 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38608822399999998, 2024-09-14T07:00:51.234 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180775 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627876+0000", 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.235 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627876+0000", 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627877+0000", 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1739999999999999e-06 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627882+0000", 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3499999999999996e-06 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.236 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627900+0000", 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7782e-05 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627914+0000", 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3653e-05 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628085+0000", 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000170884 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.237 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628099+0000", 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4610000000000001e-05 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172376 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851135+0000", 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161627897, 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.162856296, 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.238 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172376 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851135+0000", 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851135+0000", 2024-09-14T07:00:51.239 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851135+0000", 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2600000000000001e-07 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851136+0000", 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3209999999999999e-06 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.240 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851139+0000", 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0860000000000002e-06 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851143+0000", 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4759999999999998e-06 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.241 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851499+0000", 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000356187 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851503+0000", 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4649999999999999e-06 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596990+0000", 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548751000000002 2024-09-14T07:00:51.242 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596994+0000", 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3170000000000003e-06 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172910 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592184+0000", 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42057910500000001, 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42184074799999999, 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.243 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172910 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592184+0000", 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.244 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592184+0000", 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592184+0000", 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4999999999999998e-07 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592186+0000", 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5009999999999999e-06 2024-09-14T07:00:51.245 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592189+0000", 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9600000000000002e-06 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592193+0000", 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8079999999999998e-06 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598040+0000", 2024-09-14T07:00:51.246 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0058472100000000003 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598049+0000", 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0520000000000002e-06 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174427 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592662+0000", 2024-09-14T07:00:51.247 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42010081900000001, 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.421389242, 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174427 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592662+0000", 2024-09-14T07:00:51.248 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592662+0000", 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592662+0000", 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0400000000000002e-07 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.249 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592663+0000", 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592666+0000", 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1839999999999999e-06 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592669+0000", 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0860000000000002e-06 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.250 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599141+0000", 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006471554 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599153+0000", 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.222e-05 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175228 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.251 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593546+0000", 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41921699099999998, 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42053329699999997, 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175228 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.252 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593546+0000", 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593546+0000", 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593546+0000", 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9900000000000002e-07 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.253 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593547+0000", 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3969999999999999e-06 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593552+0000", 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5800000000000002e-06 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593556+0000", 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0389999999999998e-06 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.254 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600261+0000", 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067046320000000003 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600269+0000", 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6980000000000004e-06 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.255 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175656 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594401+0000", 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41836169200000001, 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41970569000000002, 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175656 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.256 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594401+0000", 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594401+0000", 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594402+0000", 2024-09-14T07:00:51.257 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1999999999999999e-07 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594404+0000", 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9520000000000001e-06 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594409+0000", 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8039999999999998e-06 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.258 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594416+0000", 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4479999999999999e-06 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601338+0000", 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069220139999999998 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601345+0000", 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6760000000000007e-06 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.259 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180777 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.627951+0000", 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38481195699999998, 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38618274899999999, 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.260 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180777 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627951+0000", 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627951+0000", 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.261 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627952+0000", 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.24e-06 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627957+0000", 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3360000000000002e-06 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627975+0000", 2024-09-14T07:00:51.262 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7799999999999999e-05 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.627988+0000", 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2563e-05 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628156+0000", 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016883499999999999 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.263 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628164+0000", 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8050000000000007e-06 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172377 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851161+0000", 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161601485, 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.162999898, 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.264 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172377 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851161+0000", 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851161+0000", 2024-09-14T07:00:51.265 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851161+0000", 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7700000000000001e-07 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851163+0000", 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.544e-06 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851171+0000", 2024-09-14T07:00:51.266 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4940000000000003e-06 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851175+0000", 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3500000000000001e-06 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851533+0000", 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00035798699999999999 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.267 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851540+0000", 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9670000000000004e-06 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597025+0000", 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745485328 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597033+0000", 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7020000000000001e-06 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.268 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172912 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592197+0000", 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.420565305, 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.421995018, 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.269 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172912 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592197+0000", 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592197+0000", 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.270 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592198+0000", 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6100000000000002e-07 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592199+0000", 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.234e-06 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592202+0000", 2024-09-14T07:00:51.271 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.19e-06 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592205+0000", 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7209999999999999e-06 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598083+0000", 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.005877915 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.272 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598090+0000", 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.759e-06 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174429 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592673+0000", 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42008965700000001, 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42154747799999998, 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.273 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174429 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592673+0000", 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592673+0000", 2024-09-14T07:00:51.274 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592673+0000", 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.22e-07 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592675+0000", 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4500000000000001e-06 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.275 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592680+0000", 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2449999999999998e-06 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592685+0000", 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8019999999999999e-06 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599188+0000", 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065037410000000004 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.276 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599195+0000", 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6109999999999999e-06 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175229 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593569+0000", 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41919317299999997, 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42067883700000003, 2024-09-14T07:00:51.277 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175229 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593569+0000", 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.278 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593569+0000", 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593570+0000", 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9999999999999997e-07 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593576+0000", 2024-09-14T07:00:51.279 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8680000000000004e-06 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593584+0000", 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2630000000000003e-06 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593591+0000", 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3110000000000002e-06 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.280 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600294+0000", 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067031130000000001 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600297+0000", 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6490000000000001e-06 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175657 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.281 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594423+0000", 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41834001100000001, 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41985218699999999, 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175657 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.282 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594423+0000", 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594423+0000", 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594423+0000", 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.96e-07 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.283 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594425+0000", 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8780000000000001e-06 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594432+0000", 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8399999999999997e-06 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594436+0000", 2024-09-14T07:00:51.284 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3499999999999999e-06 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601369+0000", 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069333800000000003 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601373+0000", 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.743e-06 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.285 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180667 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621068+0000", 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39169425299999999, 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39323564700000002, 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180667 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.286 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621068+0000", 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621068+0000", 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621069+0000", 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4200000000000002e-07 2024-09-14T07:00:51.287 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621070+0000", 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1650000000000001e-06 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621073+0000", 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.275e-06 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621079+0000", 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8850000000000004e-06 2024-09-14T07:00:51.288 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621281+0000", 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020191799999999999 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621292+0000", 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1426999999999999e-05 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180779 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628011+0000", 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38475209500000002, 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.386320094, 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180779 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628011+0000", 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628011+0000", 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628012+0000", 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1000000000000001e-06 2024-09-14T07:00:51.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628017+0000", 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3859999999999998e-06 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628046+0000", 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9104000000000001e-05 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628060+0000", 2024-09-14T07:00:51.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3942e-05 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628191+0000", 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013098 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628195+0000", 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172378 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851189+0000", 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1615734849999999, 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.163169962, 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172378 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851189+0000", 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851189+0000", 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851189+0000", 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8000000000000002e-07 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.295 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851191+0000", 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4929999999999999e-06 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851195+0000", 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8449999999999996e-06 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851197+0000", 2024-09-14T07:00:51.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6079999999999998e-06 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851564+0000", 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00036624100000000001 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851567+0000", 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3009999999999998e-06 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597061+0000", 2024-09-14T07:00:51.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549408900000003 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597065+0000", 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0369999999999996e-06 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172914 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.298 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592208+0000", 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42055466899999999, 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42223310400000003, 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172914 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592208+0000", 2024-09-14T07:00:51.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592208+0000", 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592208+0000", 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592209+0000", 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2020000000000001e-06 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592213+0000", 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1549999999999999e-06 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592215+0000", 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7389999999999999e-06 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598126+0000", 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0059107489999999999 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598134+0000", 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.695e-06 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.302 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174431 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592701+0000", 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42006168700000002, 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42176796599999999, 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174431 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592701+0000", 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592701+0000", 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592702+0000", 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6300000000000005e-07 2024-09-14T07:00:51.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592704+0000", 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5179999999999999e-06 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592710+0000", 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9689999999999999e-06 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592715+0000", 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9470000000000002e-06 2024-09-14T07:00:51.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599230+0000", 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065148860000000001 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599238+0000", 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0290000000000005e-06 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175230 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593597+0000", 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41916559399999997, 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42090016899999999, 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175230 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593597+0000", 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593597+0000", 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593597+0000", 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.03e-07 2024-09-14T07:00:51.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593599+0000", 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0049999999999999e-06 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593606+0000", 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5760000000000002e-06 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593612+0000", 2024-09-14T07:00:51.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.409e-06 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600327+0000", 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006714172 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600334+0000", 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5789999999999998e-06 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175659 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594450+0000", 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41831225599999999, 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42007369300000003, 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175659 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.311 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594450+0000", 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594450+0000", 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.312 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594451+0000", 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594452+0000", 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5200000000000001e-06 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594459+0000", 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.148e-06 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594467+0000", 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5120000000000002e-06 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601402+0000", 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069351539999999998 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601410+0000", 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4660000000000002e-06 2024-09-14T07:00:51.314 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180781 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628073+0000", 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38468954999999999, 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38647867699999999, 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.315 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180781 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628073+0000", 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628073+0000", 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628074+0000", 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3699999999999999e-07 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628087+0000", 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2893e-05 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628100+0000", 2024-09-14T07:00:51.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3614e-05 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628126+0000", 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5741999999999998e-05 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628231+0000", 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104969 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.318 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628239+0000", 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0309999999999998e-06 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172407 3.8 3.602f9a08 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851201+0000", 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1615618560000001, 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.163378748, 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.319 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172407 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851201+0000", 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851201+0000", 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851201+0000", 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851202+0000", 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.263e-06 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851205+0000", 2024-09-14T07:00:51.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.134e-06 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851208+0000", 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.559e-06 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851596+0000", 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038847000000000003 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851604+0000", 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.075e-06 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597097+0000", 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549312000000001 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597104+0000", 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7309999999999993e-06 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.323 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172916 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592219+0000", 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42054413800000001, 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42239227000000001, 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172916 2024-09-14T07:00:51.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592219+0000", 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592219+0000", 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592219+0000", 2024-09-14T07:00:51.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2099999999999998e-07 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592220+0000", 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5999999999999999e-06 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592224+0000", 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.157e-06 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592226+0000", 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5600000000000001e-06 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598160+0000", 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0059334690000000002 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598163+0000", 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8659999999999999e-06 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.327 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174433 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592721+0000", 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42004139600000001, 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.421917247, 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.328 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174433 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592721+0000", 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592721+0000", 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592722+0000", 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.15e-07 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592724+0000", 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0360000000000001e-06 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592730+0000", 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.229e-06 2024-09-14T07:00:51.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592734+0000", 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1069999999999998e-06 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599264+0000", 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065301719999999999 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599268+0000", 2024-09-14T07:00:51.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.777e-06 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175231 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593630+0000", 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41913233, 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.421034835, 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.332 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175231 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593630+0000", 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593630+0000", 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593631+0000", 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9699999999999996e-07 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593633+0000", 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.119e-06 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593639+0000", 2024-09-14T07:00:51.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9259999999999997e-06 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593647+0000", 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4330000000000004e-06 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600361+0000", 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067136660000000001 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600365+0000", 2024-09-14T07:00:51.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8079999999999998e-06 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175660 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594489+0000", 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41827339200000002, 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42020380699999998, 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.336 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175660 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594489+0000", 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594489+0000", 2024-09-14T07:00:51.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594490+0000", 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594492+0000", 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3089999999999998e-06 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594498+0000", 2024-09-14T07:00:51.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9190000000000004e-06 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594507+0000", 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8599999999999999e-06 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601434+0000", 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069267679999999998 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601442+0000", 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0539999999999995e-06 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180669 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621105+0000", 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39165799499999998, 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39361603099999998, 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.340 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180669 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621105+0000", 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621105+0000", 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621105+0000", 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7100000000000002e-07 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621107+0000", 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1040000000000001e-06 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621114+0000", 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7059999999999998e-06 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621118+0000", 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5199999999999999e-06 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621326+0000", 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020776 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621335+0000", 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5529999999999993e-06 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.344 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180783 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628133+0000", 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38462987199999998, 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38661452200000002, 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180783 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628133+0000", 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628133+0000", 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628133+0000", 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0400000000000002e-07 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628135+0000", 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2529999999999999e-06 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628142+0000", 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.567e-06 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628147+0000", 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0370000000000002e-06 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628263+0000", 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000116013 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628267+0000", 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8089999999999999e-06 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.348 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172423 3.8 3.f44df728 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851226+0000", 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1615365, 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1635488709999999, 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172423 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851226+0000", 2024-09-14T07:00:51.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851226+0000", 2024-09-14T07:00:51.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851226+0000", 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851228+0000", 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.418e-06 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851232+0000", 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.755e-06 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851235+0000", 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.118e-06 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851628+0000", 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00039314500000000002 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851631+0000", 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2990000000000001e-06 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597128+0000", 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549664999999998 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597132+0000", 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7479999999999999e-06 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172917 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.355 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592237+0000", 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42052573500000001, 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42257246100000001, 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172917 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592237+0000", 2024-09-14T07:00:51.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592237+0000", 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592237+0000", 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4799999999999999e-07 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592239+0000", 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5880000000000001e-06 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592243+0000", 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8920000000000002e-06 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592246+0000", 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7470000000000001e-06 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598195+0000", 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0059491889999999997 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598202+0000", 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6860000000000006e-06 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.359 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174435 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592745+0000", 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42001803599999998, 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42209225700000003, 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174435 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592745+0000", 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592745+0000", 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592745+0000", 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8800000000000002e-07 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592747+0000", 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592754+0000", 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1119999999999998e-06 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592767+0000", 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2873e-05 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599299+0000", 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065321190000000003 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599307+0000", 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7349999999999996e-06 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.363 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175232 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593665+0000", 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41909716499999999, 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42119864899999998, 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175232 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593665+0000", 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593665+0000", 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593666+0000", 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4000000000000002e-07 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593668+0000", 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9319999999999999e-06 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593673+0000", 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4609999999999999e-06 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593682+0000", 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2470000000000008e-06 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600395+0000", 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067136749999999997 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600403+0000", 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8180000000000004e-06 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.367 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175661 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594535+0000", 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41822743800000001, 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42035936000000002, 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175661 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594535+0000", 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594535+0000", 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594536+0000", 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2399999999999999e-07 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594538+0000", 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9800000000000001e-06 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594543+0000", 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9919999999999998e-06 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594545+0000", 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8219999999999998e-06 2024-09-14T07:00:51.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601472+0000", 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069262880000000001 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601479+0000", 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6199999999999999e-06 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180863 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628154+0000", 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38460821499999998, 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38676798299999998, 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180863 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628154+0000", 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628154+0000", 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628155+0000", 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9999999999999998e-07 2024-09-14T07:00:51.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628157+0000", 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8509999999999999e-06 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628164+0000", 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1029999999999996e-06 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628168+0000", 2024-09-14T07:00:51.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.386e-06 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628296+0000", 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012811100000000001 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628304+0000", 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7039999999999994e-06 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172493 3.8 3.eec2c188 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1615230560000001, 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1637094669999999, 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172493 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.376 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:51.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851241+0000", 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164e-06 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851244+0000", 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1259999999999998e-06 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851247+0000", 2024-09-14T07:00:51.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851660+0000", 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041335800000000001 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851667+0000", 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9809999999999997e-06 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597160+0000", 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549261700000002 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597167+0000", 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2629999999999997e-06 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172919 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592249+0000", 2024-09-14T07:00:51.380 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.420513568, 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42273213399999998, 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172919 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592249+0000", 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592249+0000", 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592249+0000", 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592256+0000", 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0180000000000004e-06 2024-09-14T07:00:51.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592260+0000", 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.613e-06 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592263+0000", 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3100000000000001e-06 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598227+0000", 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.005963678 2024-09-14T07:00:51.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598231+0000", 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7720000000000001e-06 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174437 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592776+0000", 2024-09-14T07:00:51.384 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41998632200000002, 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42223267799999997, 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174437 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592776+0000", 2024-09-14T07:00:51.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592776+0000", 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592777+0000", 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6199999999999998e-07 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592779+0000", 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592785+0000", 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9580000000000004e-06 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592789+0000", 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3810000000000001e-06 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599332+0000", 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065426349999999998 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599336+0000", 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7139999999999999e-06 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.388 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175233 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593689+0000", 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41907322200000002, 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42134644700000001, 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175233 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593689+0000", 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593689+0000", 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593690+0000", 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6000000000000004e-07 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593692+0000", 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9939999999999999e-06 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593699+0000", 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2509999999999997e-06 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593703+0000", 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2119999999999997e-06 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600428+0000", 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067247160000000004 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600432+0000", 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.552e-06 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175663 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.392 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594549+0000", 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41821350400000001, 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42051424700000001, 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175663 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594549+0000", 2024-09-14T07:00:51.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594549+0000", 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594549+0000", 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594561+0000", 2024-09-14T07:00:51.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1474999999999999e-05 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594568+0000", 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5220000000000001e-06 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594574+0000", 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7529999999999998e-06 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601503+0000", 2024-09-14T07:00:51.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069291600000000002 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601507+0000", 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6200000000000001e-06 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180671 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621158+0000", 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39160507300000003, 2024-09-14T07:00:51.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39393233700000002, 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180671 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621158+0000", 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621158+0000", 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621158+0000", 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3200000000000001e-07 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621159+0000", 2024-09-14T07:00:51.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621162+0000", 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2890000000000002e-06 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621165+0000", 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0000000000000001e-06 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621367+0000", 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020203699999999999 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621375+0000", 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4819999999999997e-06 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180864 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628188+0000", 2024-09-14T07:00:51.400 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.384574743, 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38692967099999998, 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180864 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628188+0000", 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628188+0000", 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628188+0000", 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.39e-07 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628190+0000", 2024-09-14T07:00:51.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.745e-06 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628194+0000", 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3919999999999996e-06 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628197+0000", 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1020000000000001e-06 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628327+0000", 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000130006 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628331+0000", 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5719999999999999e-06 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172499 3.8 3.21b36cc8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161508749, 2024-09-14T07:00:51.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.163890246, 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172499 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.22e-07 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851255+0000", 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.114e-06 2024-09-14T07:00:51.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851259+0000", 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2909999999999999e-06 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851261+0000", 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.655e-06 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851691+0000", 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043023900000000001 2024-09-14T07:00:51.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851695+0000", 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4000000000000001e-06 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597190+0000", 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745495146 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597194+0000", 2024-09-14T07:00:51.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7050000000000001e-06 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172921 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592267+0000", 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42049592000000002, 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42290971799999999, 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.409 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172921 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592267+0000", 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592267+0000", 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592267+0000", 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7099999999999997e-07 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592268+0000", 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1090000000000001e-06 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592272+0000", 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3349999999999998e-06 2024-09-14T07:00:51.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592274+0000", 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.655e-06 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598264+0000", 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0059900530000000004 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598272+0000", 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9130000000000003e-06 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174439 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592820+0000", 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41994220700000001, 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42238384600000001, 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.413 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174439 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592820+0000", 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592820+0000", 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592821+0000", 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4799999999999998e-07 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592824+0000", 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.429e-06 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592830+0000", 2024-09-14T07:00:51.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3339999999999998e-06 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592834+0000", 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5990000000000004e-06 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599374+0000", 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065391049999999999 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599381+0000", 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175234 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593709+0000", 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41905326900000001, 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42152146299999999, 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.417 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175234 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593709+0000", 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593709+0000", 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593710+0000", 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593712+0000", 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.79e-06 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593718+0000", 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.63e-06 2024-09-14T07:00:51.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593723+0000", 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9969999999999998e-06 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600462+0000", 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067392509999999999 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600470+0000", 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6760000000000007e-06 2024-09-14T07:00:51.420 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175871 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594582+0000", 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41818102899999998, 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42067668800000002, 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.421 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175871 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594582+0000", 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594582+0000", 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594582+0000", 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2799999999999996e-07 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594584+0000", 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2220000000000001e-06 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594608+0000", 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3615999999999999e-05 2024-09-14T07:00:51.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594619+0000", 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0591000000000001e-05 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601536+0000", 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069175479999999999 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601544+0000", 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3799999999999996e-06 2024-09-14T07:00:51.424 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180865 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628204+0000", 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.384558768, 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.387080798, 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.425 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180865 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628204+0000", 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628204+0000", 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628204+0000", 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7099999999999997e-07 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628206+0000", 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7990000000000001e-06 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628212+0000", 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7509999999999997e-06 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628218+0000", 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7540000000000003e-06 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628360+0000", 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014264400000000001 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628368+0000", 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6690000000000006e-06 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172507 3.8 3.fa9413e8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161498197, 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164047896, 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.429 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172507 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:00:51.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851265+0000", 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851266+0000", 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851269+0000", 2024-09-14T07:00:51.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0079999999999998e-06 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851271+0000", 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.531e-06 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851725+0000", 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00045350300000000001 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851732+0000", 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9600000000000003e-06 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597223+0000", 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745490926 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597230+0000", 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4499999999999998e-06 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172923 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592288+0000", 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42047494299999999, 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.423056818, 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172923 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592288+0000", 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592288+0000", 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592288+0000", 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6499999999999999e-07 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592290+0000", 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3960000000000001e-06 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592294+0000", 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3100000000000002e-06 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592297+0000", 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:00:51.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598300+0000", 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00600324 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598304+0000", 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.771e-06 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.437 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174441 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592842+0000", 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41992102999999997, 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42252939900000003, 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174441 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592842+0000", 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592842+0000", 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592842+0000", 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592844+0000", 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3539999999999998e-06 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592850+0000", 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1319999999999999e-06 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592855+0000", 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8810000000000004e-06 2024-09-14T07:00:51.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599406+0000", 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065510300000000002 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599411+0000", 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1740000000000002e-06 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175235 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593738+0000", 2024-09-14T07:00:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41902487399999999, 2024-09-14T07:00:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42166088400000001, 2024-09-14T07:00:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.442 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175235 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593738+0000", 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593738+0000", 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593738+0000", 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.99e-07 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593740+0000", 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0279999999999999e-06 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593746+0000", 2024-09-14T07:00:51.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7880000000000003e-06 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593757+0000", 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0441999999999999e-05 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600495+0000", 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067379340000000001 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600498+0000", 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7019999999999999e-06 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175872 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594624+0000", 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41813852600000001, 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42080107900000002, 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.446 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175872 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594624+0000", 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594624+0000", 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594624+0000", 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594626+0000", 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7600000000000001e-06 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594631+0000", 2024-09-14T07:00:51.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4390000000000002e-06 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594633+0000", 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.491e-06 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601571+0000", 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069383090000000001 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601576+0000", 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5800000000000002e-06 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180673 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621207+0000", 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39155614399999999, 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39424610199999999, 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.450 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180673 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621207+0000", 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621207+0000", 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621207+0000", 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621209+0000", 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.902e-06 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621215+0000", 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7400000000000001e-06 2024-09-14T07:00:51.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621218+0000", 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2780000000000002e-06 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621410+0000", 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000192151 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621419+0000", 2024-09-14T07:00:51.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6179999999999995e-06 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180866 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628225+0000", 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38453723899999998, 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38725531200000002, 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.454 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180866 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628225+0000", 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628225+0000", 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628226+0000", 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5600000000000001e-07 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628228+0000", 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8360000000000001e-06 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628233+0000", 2024-09-14T07:00:51.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5140000000000001e-06 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628239+0000", 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5029999999999997e-06 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628394+0000", 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015518199999999999 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628399+0000", 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8450000000000002e-06 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172606 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1614879890000001, 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1642327260000001, 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.458 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172606 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:51.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851276+0000", 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9399999999999993e-07 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851279+0000", 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0070000000000002e-06 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851282+0000", 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9210000000000001e-06 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851756+0000", 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000474005 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851759+0000", 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3670000000000001e-06 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597256+0000", 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549659599999996 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597260+0000", 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:51.462 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173023 3.8 3.6a55dc88 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592301+0000", 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42046186800000002, 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42323852200000001, 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.463 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173023 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592301+0000", 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592301+0000", 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592301+0000", 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7e-07 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592302+0000", 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0359999999999999e-06 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592305+0000", 2024-09-14T07:00:51.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2509999999999999e-06 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592308+0000", 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6240000000000002e-06 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598334+0000", 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0060263729999999998 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598342+0000", 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9719999999999992e-06 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174443 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592864+0000", 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41989900800000002, 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42270211499999999, 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.467 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174443 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592864+0000", 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592864+0000", 2024-09-14T07:00:51.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592864+0000", 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7999999999999995e-07 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592866+0000", 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9420000000000002e-06 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592879+0000", 2024-09-14T07:00:51.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2843000000000001e-05 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592884+0000", 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3739999999999997e-06 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599440+0000", 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065552939999999997 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599447+0000", 2024-09-14T07:00:51.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7000000000000008e-06 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175434 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593765+0000", 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.418997709, 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42182957599999998, 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.471 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175434 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593765+0000", 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593765+0000", 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593765+0000", 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1600000000000001e-07 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593768+0000", 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2469999999999998e-06 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593776+0000", 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7959999999999996e-06 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593786+0000", 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0713999999999999e-05 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600528+0000", 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067413330000000004 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600535+0000", 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5429999999999996e-06 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.474 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175873 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594637+0000", 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41812568500000002, 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42098532300000002, 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.475 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175873 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594637+0000", 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594637+0000", 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594637+0000", 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.16e-07 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594638+0000", 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594642+0000", 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0899999999999998e-06 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594647+0000", 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2010000000000001e-06 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601606+0000", 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069595250000000003 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601614+0000", 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4379999999999998e-06 2024-09-14T07:00:51.478 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180479 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615392+0000", 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.397370956, 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400257215, 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.479 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180479 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615392+0000", 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615392+0000", 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615392+0000", 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4499999999999998e-07 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615393+0000", 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9000000000000005e-07 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615396+0000", 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1140000000000002e-06 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615399+0000", 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.083e-06 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615555+0000", 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000156181 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615563+0000", 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0490000000000004e-06 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.482 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180877 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628260+0000", 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38450290300000001, 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38741706999999997, 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180877 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.483 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628260+0000", 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628260+0000", 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628260+0000", 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T07:00:51.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628262+0000", 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.689e-06 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628266+0000", 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3819999999999997e-06 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628269+0000", 2024-09-14T07:00:51.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8789999999999999e-06 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628438+0000", 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000169216 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628447+0000", 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9080000000000001e-06 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172607 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161470703, 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164411614, 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.487 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.488 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.488 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172607 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7000000000000001e-07 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851294+0000", 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3710000000000001e-06 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851297+0000", 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7850000000000002e-06 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851309+0000", 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1148000000000001e-05 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851788+0000", 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00047951899999999997 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851803+0000", 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5074e-05 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597290+0000", 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548693700000002 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597298+0000", 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3590000000000008e-06 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.492 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173115 3.8 3.1fc87308 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592311+0000", 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42045154000000001, 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.423424676, 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173115 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592311+0000", 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592311+0000", 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592311+0000", 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592312+0000", 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.048e-06 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592316+0000", 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592318+0000", 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6249999999999999e-06 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598367+0000", 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0060484780000000004 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598371+0000", 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4700000000000004e-06 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.496 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174445 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592892+0000", 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.419870837, 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42287198500000001, 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174445 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592892+0000", 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592892+0000", 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592892+0000", 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.15e-07 2024-09-14T07:00:51.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592903+0000", 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0392e-05 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592910+0000", 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592916+0000", 2024-09-14T07:00:51.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599471+0000", 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065554300000000001 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599476+0000", 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0690000000000001e-06 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175436 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593816+0000", 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41894637600000001, 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42197416399999998, 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175436 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.501 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593816+0000", 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593816+0000", 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593817+0000", 2024-09-14T07:00:51.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7800000000000002e-07 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593819+0000", 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5409999999999999e-06 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593827+0000", 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6529999999999994e-06 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593832+0000", 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9049999999999996e-06 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600559+0000", 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067273630000000001 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600563+0000", 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7720000000000001e-06 2024-09-14T07:00:51.504 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175874 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594651+0000", 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41811184899999998, 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42119994799999999, 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.505 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175874 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594651+0000", 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594651+0000", 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594651+0000", 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6000000000000004e-07 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594653+0000", 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.728e-06 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594659+0000", 2024-09-14T07:00:51.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5010000000000004e-06 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594663+0000", 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4100000000000001e-06 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601637+0000", 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069742179999999999 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601641+0000", 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7290000000000002e-06 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180481 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615418+0000", 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39734446699999998, 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40046065800000002, 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.509 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180481 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615418+0000", 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615418+0000", 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615419+0000", 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0999999999999999e-07 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615421+0000", 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9589999999999998e-06 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615427+0000", 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9819999999999996e-06 2024-09-14T07:00:51.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615431+0000", 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.921e-06 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615600+0000", 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00016853300000000001 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615608+0000", 2024-09-14T07:00:51.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7409999999999992e-06 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180675 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621239+0000", 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39152380799999997, 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.394669875, 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.513 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180675 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621239+0000", 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621239+0000", 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621239+0000", 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.23e-07 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621240+0000", 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.065e-06 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621243+0000", 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621246+0000", 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.548e-06 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621457+0000", 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000210709 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621464+0000", 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5920000000000003e-06 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180878 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628275+0000", 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.384487307, 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38766131700000001, 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180878 2024-09-14T07:00:51.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628275+0000", 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628275+0000", 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628276+0000", 2024-09-14T07:00:51.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628277+0000", 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7630000000000001e-06 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628283+0000", 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8560000000000003e-06 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628288+0000", 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4259999999999996e-06 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628473+0000", 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018478900000000001 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628480+0000", 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9779999999999999e-06 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172608 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161442358, 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1646457480000001, 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172608 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.521 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851321+0000", 2024-09-14T07:00:51.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0600000000000001e-07 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851322+0000", 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.751e-06 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851331+0000", 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1799999999999996e-06 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851334+0000", 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3409999999999999e-06 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851830+0000", 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00049571000000000001 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851833+0000", 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6069999999999999e-06 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597326+0000", 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549257700000005 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597333+0000", 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173123 3.8 3.afd79e08 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.525 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592332+0000", 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42043080700000002, 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42376420300000001, 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173123 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592332+0000", 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592332+0000", 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592332+0000", 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5600000000000001e-07 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592334+0000", 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3799999999999999e-06 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592338+0000", 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8500000000000004e-06 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592340+0000", 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7590000000000002e-06 2024-09-14T07:00:51.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598401+0000", 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0060604259999999998 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598409+0000", 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5939999999999994e-06 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.529 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174522 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592927+0000", 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41983613199999997, 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.423201298, 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174522 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592927+0000", 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592927+0000", 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592927+0000", 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9800000000000003e-07 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592929+0000", 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.08e-06 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592936+0000", 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3500000000000002e-06 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592940+0000", 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8489999999999997e-06 2024-09-14T07:00:51.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599509+0000", 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065685650000000002 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599517+0000", 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7449999999999995e-06 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175438 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593850+0000", 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41891302699999999, 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42231003700000003, 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175438 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593850+0000", 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593850+0000", 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593850+0000", 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9100000000000004e-07 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593853+0000", 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.145e-06 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593861+0000", 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3030000000000002e-06 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593867+0000", 2024-09-14T07:00:51.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.737e-06 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600592+0000", 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067244879999999998 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600599+0000", 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5920000000000003e-06 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175875 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594669+0000", 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41809415, 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42153547699999999, 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175875 2024-09-14T07:00:51.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594669+0000", 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594669+0000", 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594669+0000", 2024-09-14T07:00:51.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594671+0000", 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7129999999999999e-06 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594677+0000", 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.708e-06 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594683+0000", 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4890000000000003e-06 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601671+0000", 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069878589999999999 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601678+0000", 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5889999999999997e-06 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180483 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615438+0000", 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39732500100000001, 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40079432399999998, 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180483 2024-09-14T07:00:51.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615438+0000", 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615438+0000", 2024-09-14T07:00:51.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615438+0000", 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6199999999999998e-07 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615440+0000", 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7799999999999999e-06 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615446+0000", 2024-09-14T07:00:51.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8610000000000003e-06 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615450+0000", 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.827e-06 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615641+0000", 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019179700000000001 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615649+0000", 2024-09-14T07:00:51.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2810000000000003e-06 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180879 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628295+0000", 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38446803400000001, 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38796617700000002, 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.546 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180879 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628295+0000", 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628295+0000", 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628295+0000", 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4900000000000001e-07 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628297+0000", 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.75e-06 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628304+0000", 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.905e-06 2024-09-14T07:00:51.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628308+0000", 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1840000000000001e-06 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628508+0000", 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000200158 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628516+0000", 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8280000000000003e-06 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172609 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161422712, 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164947738, 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172609 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:51.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9200000000000002e-07 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851342+0000", 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.451e-06 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851346+0000", 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9999999999999998e-06 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851348+0000", 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5780000000000001e-06 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851868+0000", 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051942399999999999 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851876+0000", 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6850000000000007e-06 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597371+0000", 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745494502 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597378+0000", 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6129999999999998e-06 2024-09-14T07:00:51.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173157 3.8 3.f8e5c3e8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592344+0000", 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42041886299999998, 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42397614099999997, 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.555 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173157 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592344+0000", 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592344+0000", 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592344+0000", 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6600000000000002e-07 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592345+0000", 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.049e-06 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592348+0000", 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1669999999999999e-06 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592351+0000", 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.633e-06 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598433+0000", 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0060821510000000001 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598437+0000", 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7840000000000001e-06 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174523 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592955+0000", 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41980752799999999, 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42339136300000002, 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174523 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.559 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592955+0000", 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592955+0000", 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592956+0000", 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.27e-07 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592958+0000", 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.069e-06 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592964+0000", 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3169999999999998e-06 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592971+0000", 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6109999999999999e-06 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599543+0000", 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006572909 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599547+0000", 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7349999999999998e-06 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175440 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593879+0000", 2024-09-14T07:00:51.562 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41888368199999998, 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.422495177, 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175440 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593879+0000", 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593879+0000", 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593880+0000", 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1500000000000004e-07 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593882+0000", 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2879999999999999e-06 2024-09-14T07:00:51.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593888+0000", 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5420000000000002e-06 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593894+0000", 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8860000000000001e-06 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600623+0000", 2024-09-14T07:00:51.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067288690000000002 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600629+0000", 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6570000000000002e-06 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175876 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594703+0000", 2024-09-14T07:00:51.566 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41806010199999999, 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42169936699999999, 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175876 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594703+0000", 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594703+0000", 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594703+0000", 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.65e-07 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594704+0000", 2024-09-14T07:00:51.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2109999999999999e-06 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594711+0000", 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6989999999999997e-06 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594719+0000", 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7250000000000003e-06 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601701+0000", 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006981737 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601705+0000", 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7809999999999999e-06 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180485 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615460+0000", 2024-09-14T07:00:51.570 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.397302288, 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.400967992, 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180485 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615460+0000", 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615460+0000", 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615461+0000", 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7899999999999999e-07 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615466+0000", 2024-09-14T07:00:51.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6869999999999999e-06 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615471+0000", 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0940000000000001e-06 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615473+0000", 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6599999999999999e-06 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615689+0000", 2024-09-14T07:00:51.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021562400000000001 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615698+0000", 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0380000000000005e-06 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180677 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621283+0000", 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.391479784, 2024-09-14T07:00:51.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39517318699999998, 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180677 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621283+0000", 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621283+0000", 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621283+0000", 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621285+0000", 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621291+0000", 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6160000000000001e-06 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621296+0000", 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4210000000000003e-06 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621499+0000", 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020324300000000001 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621525+0000", 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5108000000000001e-05 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180880 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628326+0000", 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38443690400000002, 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38815697900000001, 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.578 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180880 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628326+0000", 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628326+0000", 2024-09-14T07:00:51.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628326+0000", 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2500000000000001e-07 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628328+0000", 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.621e-06 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628332+0000", 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.313e-06 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628337+0000", 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6689999999999997e-06 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628540+0000", 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020370800000000001 2024-09-14T07:00:51.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628544+0000", 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6330000000000002e-06 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172610 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161401908, 2024-09-14T07:00:51.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1651500829999999, 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172610 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6800000000000002e-07 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851362+0000", 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4300000000000001e-06 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851366+0000", 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.811e-06 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851369+0000", 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4739999999999999e-06 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851901+0000", 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053255399999999997 2024-09-14T07:00:51.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851905+0000", 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4170000000000001e-06 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597404+0000", 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549902099999998 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597408+0000", 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.527e-06 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.586 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173382 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592354+0000", 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42040865799999999, 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.424188291, 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173382 2024-09-14T07:00:51.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592354+0000", 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592354+0000", 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592354+0000", 2024-09-14T07:00:51.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592355+0000", 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.017e-06 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592358+0000", 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.101e-06 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592361+0000", 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.464e-06 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598466+0000", 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061052010000000002 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598474+0000", 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6129999999999998e-06 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174524 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592984+0000", 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41977881099999997, 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42358525200000002, 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174524 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592984+0000", 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592984+0000", 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592984+0000", 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2200000000000004e-07 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592986+0000", 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0279999999999999e-06 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592992+0000", 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8279999999999999e-06 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592999+0000", 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5880000000000003e-06 2024-09-14T07:00:51.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599577+0000", 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065783270000000001 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599585+0000", 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5109999999999997e-06 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175454 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593908+0000", 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41885482600000001, 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42268939900000002, 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175454 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593908+0000", 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593908+0000", 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593908+0000", 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593910+0000", 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1280000000000002e-06 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593917+0000", 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2020000000000001e-06 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593920+0000", 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8290000000000001e-06 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600660+0000", 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067393840000000002 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600667+0000", 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0400000000000004e-06 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175877 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594735+0000", 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41802788499999999, 2024-09-14T07:00:51.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42188906300000001, 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175877 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594735+0000", 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.599 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594735+0000", 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594735+0000", 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4899999999999995e-07 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594737+0000", 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594744+0000", 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0379999999999996e-06 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594754+0000", 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0284e-05 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601736+0000", 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0069820680000000001 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601743+0000", 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5259999999999996e-06 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177695 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596447+0000", 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41631549800000001, 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42020463299999999, 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.602 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177695 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596447+0000", 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596447+0000", 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596447+0000", 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596448+0000", 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8299999999999995e-07 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596453+0000", 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1209999999999998e-06 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596456+0000", 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5969999999999999e-06 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601858+0000", 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054021770000000002 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601863+0000", 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8960000000000002e-06 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.605 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180487 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615477+0000", 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.397285263, 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40120080899999999, 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.606 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180487 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615477+0000", 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615477+0000", 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615478+0000", 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2399999999999999e-07 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615479+0000", 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.11e-06 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615483+0000", 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4299999999999999e-06 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615491+0000", 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5889999999999997e-06 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615737+0000", 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00024638600000000002 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615745+0000", 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6590000000000007e-06 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.609 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180881 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628351+0000", 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38441148200000003, 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38835466699999999, 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180881 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.610 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628351+0000", 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628351+0000", 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628352+0000", 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2399999999999999e-07 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628353+0000", 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8929999999999999e-06 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628360+0000", 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2160000000000003e-06 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628365+0000", 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628574+0000", 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020955600000000001 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628582+0000", 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1729999999999995e-06 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172611 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.613 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.16138363, 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1653545789999999, 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172611 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.79e-07 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851381+0000", 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7379999999999999e-06 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851385+0000", 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5750000000000001e-06 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851387+0000", 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851934+0000", 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00054657299999999996 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851942+0000", 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2549999999999997e-06 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597437+0000", 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549549800000003 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597445+0000", 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5839999999999997e-06 2024-09-14T07:00:51.617 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173383 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592375+0000", 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42038784899999998, 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42438941200000002, 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.618 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173383 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592375+0000", 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592375+0000", 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592375+0000", 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6600000000000002e-07 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592377+0000", 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3799999999999999e-06 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592381+0000", 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6800000000000001e-06 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592384+0000", 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7489999999999999e-06 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598498+0000", 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006114151 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598502+0000", 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7890000000000001e-06 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.621 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174525 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593006+0000", 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41975647999999999, 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42378606400000002, 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.622 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174525 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593006+0000", 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593006+0000", 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593007+0000", 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1800000000000001e-07 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593009+0000", 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3489999999999999e-06 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593019+0000", 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.043e-05 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593025+0000", 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3720000000000004e-06 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599609+0000", 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065844359999999999 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599613+0000", 2024-09-14T07:00:51.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8279999999999996e-06 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175455 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593940+0000", 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41882253699999999, 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42288212800000002, 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.626 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175455 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593940+0000", 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593940+0000", 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593941+0000", 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-07 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593948+0000", 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4760000000000001e-06 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593956+0000", 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6529999999999994e-06 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593960+0000", 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4800000000000003e-06 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600693+0000", 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067323319999999997 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.629 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600696+0000", 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7799999999999998e-06 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175878 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594761+0000", 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41800157199999999, 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42208950899999997, 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.630 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175878 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594761+0000", 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594761+0000", 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594761+0000", 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.39e-07 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594764+0000", 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1969999999999999e-06 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594768+0000", 2024-09-14T07:00:51.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0130000000000004e-06 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594770+0000", 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7199999999999998e-06 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601767+0000", 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006996866 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601771+0000", 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6650000000000001e-06 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177696 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596470+0000", 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41629262500000003, 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42040872899999998, 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.634 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177696 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596470+0000", 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596470+0000", 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596470+0000", 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596472+0000", 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596478+0000", 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7150000000000003e-06 2024-09-14T07:00:51.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596483+0000", 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7430000000000002e-06 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601895+0000", 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054119349999999997 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601902+0000", 2024-09-14T07:00:51.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7530000000000001e-06 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180489 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615504+0000", 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39725908199999999, 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40140161800000002, 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.638 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180489 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615504+0000", 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615504+0000", 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615504+0000", 2024-09-14T07:00:51.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3e-07 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615506+0000", 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0499999999999999e-06 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615512+0000", 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0700000000000003e-06 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615517+0000", 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8040000000000001e-06 2024-09-14T07:00:51.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615779+0000", 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026230200000000001 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615784+0000", 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5419999999999999e-06 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180679 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621317+0000", 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39144613900000003, 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39561660500000001, 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180679 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.642 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621317+0000", 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621317+0000", 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621317+0000", 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4900000000000002e-07 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621318+0000", 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.172e-06 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621321+0000", 2024-09-14T07:00:51.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1820000000000002e-06 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621324+0000", 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5059999999999998e-06 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621559+0000", 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000235469 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621567+0000", 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4869999999999996e-06 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180882 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628371+0000", 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38439171, 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.388588765, 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.646 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180882 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628371+0000", 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628371+0000", 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628371+0000", 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.22e-07 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628373+0000", 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8190000000000001e-06 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628379+0000", 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628383+0000", 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3259999999999997e-06 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628602+0000", 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00021873500000000001 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628606+0000", 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3449999999999999e-06 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172612 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1613720329999999, 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1655979599999999, 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.650 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172612 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851392+0000", 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T07:00:51.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851395+0000", 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2449999999999998e-06 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851398+0000", 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4660000000000002e-06 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851978+0000", 2024-09-14T07:00:51.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058031599999999999 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851984+0000", 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1319999999999999e-06 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597469+0000", 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548469799999995 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597473+0000", 2024-09-14T07:00:51.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8510000000000001e-06 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173384 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592389+0000", 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42037403299999998, 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42463169699999997, 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.655 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173384 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592389+0000", 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592389+0000", 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592389+0000", 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1899999999999998e-07 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592390+0000", 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0440000000000001e-06 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592393+0000", 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1580000000000001e-06 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592396+0000", 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5129999999999999e-06 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598533+0000", 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061373859999999999 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598541+0000", 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6299999999999998e-06 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174526 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593032+0000", 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41973067200000003, 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42401489799999997, 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.659 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174526 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593032+0000", 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593032+0000", 2024-09-14T07:00:51.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593033+0000", 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5300000000000004e-07 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593035+0000", 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0279999999999999e-06 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593040+0000", 2024-09-14T07:00:51.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6949999999999997e-06 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593046+0000", 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7939999999999999e-06 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599642+0000", 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0065959490000000003 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599650+0000", 2024-09-14T07:00:51.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.926e-06 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175456 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593968+0000", 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41879493899999998, 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42310663199999998, 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.663 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175456 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593968+0000", 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593968+0000", 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593968+0000", 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.08e-07 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593970+0000", 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0190000000000001e-06 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593975+0000", 2024-09-14T07:00:51.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1850000000000003e-06 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593982+0000", 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600724+0000", 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067422549999999999 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600732+0000", 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6580000000000002e-06 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175879 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594774+0000", 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41798871500000001, 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42232697400000002, 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.667 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175879 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594774+0000", 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594774+0000", 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594774+0000", 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.17e-07 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594775+0000", 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0640000000000001e-06 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594782+0000", 2024-09-14T07:00:51.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6989999999999997e-06 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594785+0000", 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3919999999999999e-06 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601808+0000", 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0070221950000000002 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601817+0000", 2024-09-14T07:00:51.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3719999999999994e-06 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177697 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596489+0000", 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41627380800000002, 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42063982999999999, 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.671 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177697 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596489+0000", 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596489+0000", 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596489+0000", 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.44e-07 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596491+0000", 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.81e-06 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596497+0000", 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3770000000000004e-06 2024-09-14T07:00:51.673 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596504+0000", 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6930000000000007e-06 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601927+0000", 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054223630000000004 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601930+0000", 2024-09-14T07:00:51.674 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6940000000000001e-06 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180491 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615524+0000", 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39723904199999999, 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40163307999999998, 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.675 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180491 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615524+0000", 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615524+0000", 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.676 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615524+0000", 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.53e-07 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615526+0000", 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7719999999999999e-06 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615532+0000", 2024-09-14T07:00:51.677 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6599999999999998e-06 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615536+0000", 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615819+0000", 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00028381599999999999 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.678 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615827+0000", 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8350000000000004e-06 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180681 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621327+0000", 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39143562900000001, 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39585607499999997, 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.679 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180681 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621327+0000", 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621327+0000", 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.680 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621327+0000", 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621328+0000", 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.136e-06 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621331+0000", 2024-09-14T07:00:51.681 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1e-06 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621334+0000", 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.492e-06 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621602+0000", 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00026813999999999999 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.682 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621610+0000", 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8199999999999997e-06 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180883 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628388+0000", 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38437426699999999, 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.388822316, 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.683 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180883 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628388+0000", 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.684 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628388+0000", 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628389+0000", 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3799999999999999e-07 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628394+0000", 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7370000000000003e-06 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.685 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628399+0000", 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7040000000000002e-06 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628402+0000", 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1020000000000001e-06 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628629+0000", 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00022641999999999999 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.686 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628632+0000", 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8800000000000001e-06 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172117 3.8 3.df9302e8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.850791+0000", 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161971324, 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1664465989999999, 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.687 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172117 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850791+0000", 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.688 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850791+0000", 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850792+0000", 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1230000000000001e-06 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850805+0000", 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2472e-05 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.689 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850820+0000", 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4636e-05 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850825+0000", 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8329999999999999e-06 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850901+0000", 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5359999999999997e-05 2024-09-14T07:00:51.690 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850912+0000", 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.077e-05 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596265+0000", 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74535297599999994 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596274+0000", 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2329999999999995e-06 2024-09-14T07:00:51.691 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172613 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1613620600000001, 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.165868999, 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.692 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172613 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.693 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.05e-07 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851402+0000", 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.026e-06 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851405+0000", 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2440000000000002e-06 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.694 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851407+0000", 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3920000000000001e-06 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852016+0000", 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060825699999999996 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852024+0000", 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0639999999999994e-06 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.695 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597505+0000", 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548078699999998 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597513+0000", 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1659999999999994e-06 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173385 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.696 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592399+0000", 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.420363763, 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42490314699999998, 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173385 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.697 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592399+0000", 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592399+0000", 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592399+0000", 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592400+0000", 2024-09-14T07:00:51.698 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.051e-06 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592403+0000", 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2780000000000002e-06 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592406+0000", 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.655e-06 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598564+0000", 2024-09-14T07:00:51.699 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061581939999999996 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598569+0000", 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4599999999999996e-06 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174527 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593055+0000", 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.419707255, 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42427323900000002, 2024-09-14T07:00:51.700 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174527 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593055+0000", 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.701 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593055+0000", 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593056+0000", 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.84e-07 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593058+0000", 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.841e-06 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.702 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593064+0000", 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6950000000000003e-06 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593072+0000", 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9899999999999997e-06 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599675+0000", 2024-09-14T07:00:51.703 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066023999999999996 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599678+0000", 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6909999999999999e-06 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175457 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593993+0000", 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.418769644, 2024-09-14T07:00:51.704 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42336320700000002, 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175457 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593993+0000", 2024-09-14T07:00:51.705 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593993+0000", 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593994+0000", 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2600000000000002e-07 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.706 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593995+0000", 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.86e-06 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594002+0000", 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4250000000000003e-06 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594009+0000", 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.379e-06 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.707 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600758+0000", 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067490969999999999 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600762+0000", 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7189999999999999e-06 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177698 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.708 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596508+0000", 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41625416999999998, 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42087422499999999, 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177698 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.709 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596508+0000", 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596508+0000", 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596509+0000", 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.710 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596510+0000", 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.525e-06 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596514+0000", 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4850000000000001e-06 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596516+0000", 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2759999999999999e-06 2024-09-14T07:00:51.711 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601960+0000", 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054444100000000002 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601968+0000", 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4880000000000001e-06 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.712 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180493 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615546+0000", 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39721715000000002, 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40186449499999999, 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180493 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.713 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615546+0000", 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615546+0000", 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615546+0000", 2024-09-14T07:00:51.714 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6899999999999998e-07 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615547+0000", 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.471e-06 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615551+0000", 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1219999999999997e-06 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615554+0000", 2024-09-14T07:00:51.715 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.965e-06 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615853+0000", 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029854700000000003 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615857+0000", 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.002e-06 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.716 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180683 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621337+0000", 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.391425737, 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39609964199999997, 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.717 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180683 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621337+0000", 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621337+0000", 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.718 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621337+0000", 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4499999999999998e-07 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621338+0000", 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.28e-06 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621344+0000", 2024-09-14T07:00:51.719 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0760000000000002e-06 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621346+0000", 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6060000000000001e-06 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621657+0000", 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000310589 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.720 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621663+0000", 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.806e-06 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180884 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628409+0000", 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.384353422, 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38905500399999998, 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.721 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180884 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628409+0000", 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.722 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628409+0000", 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628410+0000", 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.96e-07 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628411+0000", 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7689999999999999e-06 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.723 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628421+0000", 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6660000000000003e-06 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628426+0000", 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3519999999999997e-06 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628652+0000", 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00022597399999999999 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.724 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628657+0000", 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3189999999999996e-06 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172163 3.8 3.3da63ba8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.850848+0000", 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161914259, 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1666436609999999, 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.725 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172163 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850848+0000", 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850848+0000", 2024-09-14T07:00:51.726 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850849+0000", 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1399999999999997e-07 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850851+0000", 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.903e-06 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850858+0000", 2024-09-14T07:00:51.727 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.357e-06 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850862+0000", 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.534e-06 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850949+0000", 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7634999999999997e-05 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.728 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850955+0000", 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8200000000000002e-06 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596309+0000", 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74535393400000005 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596318+0000", 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.935e-06 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.729 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172614 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.16135225, 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1661122589999999, 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172614 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.730 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.731 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851411+0000", 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851412+0000", 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4900000000000004e-07 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851415+0000", 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.942e-06 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.732 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851417+0000", 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000002e-06 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852054+0000", 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00063717699999999995 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852058+0000", 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6739999999999999e-06 2024-09-14T07:00:51.733 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597543+0000", 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548541400000001 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597548+0000", 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6140000000000002e-06 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.734 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173386 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592420+0000", 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42034312299999999, 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42513489700000001, 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173386 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.735 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592420+0000", 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592420+0000", 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592420+0000", 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:00:51.736 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592422+0000", 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7269999999999999e-06 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592425+0000", 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6449999999999998e-06 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592428+0000", 2024-09-14T07:00:51.737 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5359999999999999e-06 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598598+0000", 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061697200000000001 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598605+0000", 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5909999999999998e-06 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.738 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174528 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593083+0000", 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41968007000000002, 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42449949999999997, 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174528 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.739 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593083+0000", 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593083+0000", 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593083+0000", 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2600000000000002e-07 2024-09-14T07:00:51.740 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593085+0000", 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-06 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593091+0000", 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7869999999999998e-06 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593096+0000", 2024-09-14T07:00:51.741 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3349999999999997e-06 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599707+0000", 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006611123 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599715+0000", 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.571e-06 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.742 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175458 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594016+0000", 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41874657199999998, 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42359260700000001, 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.743 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175458 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594016+0000", 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594016+0000", 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594016+0000", 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3799999999999998e-07 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594019+0000", 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1129999999999999e-06 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594024+0000", 2024-09-14T07:00:51.745 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8109999999999999e-06 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594033+0000", 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5250000000000005e-06 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600791+0000", 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067582709999999997 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.746 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600805+0000", 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.366e-05 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177699 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596519+0000", 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.416243578, 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42111726700000002, 2024-09-14T07:00:51.747 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.750 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.750 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177699 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596519+0000", 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.751 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596519+0000", 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596519+0000", 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.79e-07 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596526+0000", 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0480000000000001e-06 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.752 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596530+0000", 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.58e-06 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596535+0000", 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5220000000000001e-06 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601996+0000", 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054613420000000001 2024-09-14T07:00:51.753 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602000+0000", 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7969999999999998e-06 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180495 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615558+0000", 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39720478999999997, 2024-09-14T07:00:51.754 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40210494000000002, 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180495 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615558+0000", 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.755 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615558+0000", 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615558+0000", 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3900000000000001e-07 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.756 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615559+0000", 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0139999999999999e-06 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615562+0000", 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2569999999999999e-06 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615565+0000", 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.356e-06 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.757 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615887+0000", 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032185699999999999 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615894+0000", 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.452e-06 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180740 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.758 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621360+0000", 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39140274800000002, 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39633148800000001, 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180740 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621360+0000", 2024-09-14T07:00:51.759 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621360+0000", 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621360+0000", 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6000000000000004e-07 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621362+0000", 2024-09-14T07:00:51.760 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9319999999999999e-06 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621368+0000", 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.728e-06 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621373+0000", 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7310000000000001e-06 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.761 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621696+0000", 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032269200000000002 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621703+0000", 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.418e-06 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180885 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.762 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628434+0000", 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38432840000000001, 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38928359899999998, 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180885 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628434+0000", 2024-09-14T07:00:51.763 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628434+0000", 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628435+0000", 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9099999999999999e-07 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.764 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628437+0000", 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8819999999999999e-06 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628443+0000", 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5200000000000003e-06 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628448+0000", 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4900000000000002e-06 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.765 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628678+0000", 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000230289 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628682+0000", 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4850000000000003e-06 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172173 3.8 3.d98c40c8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.850869+0000", 2024-09-14T07:00:51.766 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1618939619999999, 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1668770939999999, 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172173 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850869+0000", 2024-09-14T07:00:51.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850869+0000", 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850869+0000", 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850870+0000", 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3200000000000001e-06 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850874+0000", 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5710000000000002e-06 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850877+0000", 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7259999999999998e-06 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850988+0000", 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111414 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850996+0000", 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7659999999999998e-06 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596352+0000", 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74535668300000002 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596360+0000", 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9440000000000005e-06 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172615 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1613425909999999, 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1663575609999999, 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.771 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172615 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851421+0000", 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669999999999999e-06 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.773 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851425+0000", 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0110000000000002e-06 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851428+0000", 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.452e-06 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852092+0000", 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000664497 2024-09-14T07:00:51.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852099+0000", 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1199999999999996e-06 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597585+0000", 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548563599999995 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597593+0000", 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7989999999999994e-06 2024-09-14T07:00:51.775 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173387 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592437+0000", 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.420325644, 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42537403800000001, 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.776 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173387 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592437+0000", 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592437+0000", 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592437+0000", 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.46e-07 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592441+0000", 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1330000000000001e-06 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592445+0000", 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6260000000000001e-06 2024-09-14T07:00:51.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592448+0000", 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0259999999999999e-06 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598630+0000", 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061816359999999999 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598633+0000", 2024-09-14T07:00:51.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.63e-06 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174529 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593103+0000", 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41965964300000003, 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42473873499999998, 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.780 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174529 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593103+0000", 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593103+0000", 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593103+0000", 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.82e-07 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593105+0000", 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9860000000000001e-06 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593111+0000", 2024-09-14T07:00:51.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7440000000000004e-06 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593115+0000", 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0999999999999997e-06 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599739+0000", 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066236969999999996 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599743+0000", 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0330000000000002e-06 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175459 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594038+0000", 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41872502099999998, 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42383205099999999, 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.784 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175459 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594038+0000", 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594038+0000", 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594038+0000", 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6600000000000003e-07 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594039+0000", 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3799999999999999e-06 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594043+0000", 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1960000000000001e-06 2024-09-14T07:00:51.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594050+0000", 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.314e-06 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600831+0000", 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067807270000000003 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600834+0000", 2024-09-14T07:00:51.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.799e-06 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177700 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596539+0000", 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41622339000000003, 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42135668399999998, 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.788 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177700 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596539+0000", 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596539+0000", 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596540+0000", 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0400000000000002e-07 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596541+0000", 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.595e-06 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596548+0000", 2024-09-14T07:00:51.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4459999999999998e-06 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596552+0000", 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5090000000000004e-06 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602032+0000", 2024-09-14T07:00:51.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054794580000000004 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602040+0000", 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9789999999999993e-06 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179628 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.614927+0000", 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39783601200000002, 2024-09-14T07:00:51.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40299732300000002, 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179628 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614927+0000", 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614927+0000", 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614927+0000", 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614928+0000", 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:00:51.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614932+0000", 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2830000000000003e-06 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614943+0000", 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0569e-05 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614988+0000", 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5181e-05 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614997+0000", 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3989999999999993e-06 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180497 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615579+0000", 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39718395000000001, 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40237230499999999, 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.796 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180497 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615579+0000", 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615579+0000", 2024-09-14T07:00:51.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615579+0000", 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2599999999999998e-07 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615581+0000", 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8020000000000001e-06 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615587+0000", 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8490000000000002e-06 2024-09-14T07:00:51.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615591+0000", 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6689999999999997e-06 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615918+0000", 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00032709299999999998 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615922+0000", 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.613e-06 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.799 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180742 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621379+0000", 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39138395599999998, 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39659992300000002, 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180742 2024-09-14T07:00:51.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621379+0000", 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621379+0000", 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621379+0000", 2024-09-14T07:00:51.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2500000000000001e-07 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621381+0000", 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.751e-06 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621386+0000", 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5180000000000004e-06 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621390+0000", 2024-09-14T07:00:51.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6059999999999999e-06 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621727+0000", 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00033732899999999998 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621732+0000", 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3390000000000003e-06 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.803 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:181008 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628470+0000", 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38429286000000001, 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38953534699999998, 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 181008 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628470+0000", 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628470+0000", 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628470+0000", 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2800000000000003e-07 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628472+0000", 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.559e-06 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628478+0000", 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.173e-06 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628482+0000", 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3830000000000002e-06 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628703+0000", 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00022100599999999999 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628707+0000", 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5949999999999999e-06 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172189 3.8 3.cf589728 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.807 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.850899+0000", 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161863624, 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1671342710000001, 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172189 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850899+0000", 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850899+0000", 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850902+0000", 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8540000000000001e-06 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850904+0000", 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9099999999999999e-06 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850908+0000", 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850915+0000", 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5869999999999998e-06 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851023+0000", 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107524 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851027+0000", 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3329999999999998e-06 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596397+0000", 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74536968599999998 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596404+0000", 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6240000000000002e-06 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172719 3.8 3.ef193508 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1613317750000001, 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1666341410000001, 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.812 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172719 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851432+0000", 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851435+0000", 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1980000000000001e-06 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851442+0000", 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9419999999999998e-06 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852133+0000", 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069105100000000004 2024-09-14T07:00:51.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852139+0000", 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597622+0000", 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548300599999995 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597635+0000", 2024-09-14T07:00:51.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3183000000000001e-05 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173388 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592458+0000", 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42030510500000001, 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42563827199999998, 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.817 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173388 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592458+0000", 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592458+0000", 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592458+0000", 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7300000000000001e-07 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592459+0000", 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.432e-06 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592463+0000", 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9090000000000002e-06 2024-09-14T07:00:51.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592466+0000", 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598664+0000", 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061981320000000003 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598672+0000", 2024-09-14T07:00:51.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5820000000000004e-06 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174530 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593122+0000", 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41964065699999997, 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42500176499999998, 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174530 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593122+0000", 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593122+0000", 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593122+0000", 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6199999999999999e-07 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593124+0000", 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9999999999999999e-06 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593131+0000", 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3400000000000003e-06 2024-09-14T07:00:51.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593135+0000", 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.639e-06 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599774+0000", 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066381959999999999 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599781+0000", 2024-09-14T07:00:51.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7319999999999998e-06 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175548 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594056+0000", 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41870657100000003, 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42409550000000001, 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175548 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594056+0000", 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594056+0000", 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594057+0000", 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594059+0000", 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9599999999999999e-06 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594064+0000", 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8690000000000001e-06 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594069+0000", 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8110000000000002e-06 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600866+0000", 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0067963320000000004 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600873+0000", 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8290000000000008e-06 2024-09-14T07:00:51.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177701 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596558+0000", 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41620430200000003, 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42161994899999999, 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177701 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596558+0000", 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596558+0000", 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596559+0000", 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5699999999999998e-07 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596560+0000", 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5290000000000001e-06 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596572+0000", 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1885000000000001e-05 2024-09-14T07:00:51.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596577+0000", 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9420000000000003e-06 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602064+0000", 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054867199999999996 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602067+0000", 2024-09-14T07:00:51.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6289999999999999e-06 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179630 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.614950+0000", 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39781315, 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40327738499999999, 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.833 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179630 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614950+0000", 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614950+0000", 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614950+0000", 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614952+0000", 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.874e-06 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614958+0000", 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6999999999999996e-06 2024-09-14T07:00:51.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614970+0000", 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1939999999999999e-05 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615028+0000", 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8625999999999997e-05 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615037+0000", 2024-09-14T07:00:51.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8780000000000004e-06 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180579 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615597+0000", 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39716530500000002, 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40265734399999997, 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.837 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180579 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615597+0000", 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615597+0000", 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615598+0000", 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6300000000000001e-07 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615599+0000", 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6300000000000001e-06 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615605+0000", 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7030000000000003e-06 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615610+0000", 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5070000000000002e-06 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615956+0000", 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00034631100000000003 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615963+0000", 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5229999999999998e-06 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.840 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180744 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621401+0000", 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39136178799999999, 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39688300500000001, 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.841 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180744 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621401+0000", 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621401+0000", 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621401+0000", 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8900000000000001e-07 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621403+0000", 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.443e-06 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621407+0000", 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0069999999999999e-06 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621410+0000", 2024-09-14T07:00:51.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.332e-06 2024-09-14T07:00:51.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621753+0000", 2024-09-14T07:00:51.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000343514 2024-09-14T07:00:51.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621757+0000", 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8380000000000004e-06 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:181009 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628500+0000", 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.384263138, 2024-09-14T07:00:51.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.38981733299999999, 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 181009 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628500+0000", 2024-09-14T07:00:51.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628500+0000", 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628500+0000", 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9799999999999999e-07 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628502+0000", 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.871e-06 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628508+0000", 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6599999999999998e-06 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628514+0000", 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9699999999999996e-06 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628728+0000", 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000213224 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628732+0000", 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.498e-06 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172228 3.8 3.fa91b08 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.850940+0000", 2024-09-14T07:00:51.849 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161822972, 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1674084389999999, 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172228 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850940+0000", 2024-09-14T07:00:51.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850940+0000", 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850940+0000", 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2099999999999998e-07 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850945+0000", 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.758e-06 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850949+0000", 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1320000000000004e-06 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850952+0000", 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9189999999999999e-06 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851063+0000", 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000111455 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851071+0000", 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4529999999999996e-06 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596438+0000", 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74536753099999997 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596446+0000", 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5039999999999996e-06 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172725 3.8 3.aaa7a368 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161305136, 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.166924125, 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.854 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172725 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851459+0000", 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.626e-06 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851466+0000", 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8349999999999998e-06 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851469+0000", 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852192+0000", 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072263799999999995 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852201+0000", 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2129999999999997e-06 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597665+0000", 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74546438299999995 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597673+0000", 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6410000000000002e-06 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.858 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173389 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592470+0000", 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42029304099999998, 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42594505199999999, 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173389 2024-09-14T07:00:51.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592470+0000", 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592470+0000", 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592470+0000", 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.46e-07 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592471+0000", 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0440000000000001e-06 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592474+0000", 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1779999999999999e-06 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592477+0000", 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5500000000000001e-06 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598698+0000", 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0062211480000000001 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598702+0000", 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7799999999999998e-06 2024-09-14T07:00:51.862 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174532 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593152+0000", 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41961070700000003, 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42529184799999997, 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.863 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174532 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593152+0000", 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593152+0000", 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593153+0000", 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6199999999999998e-07 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593155+0000", 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0020000000000001e-06 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593161+0000", 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2140000000000001e-06 2024-09-14T07:00:51.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593168+0000", 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0999999999999998e-06 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599808+0000", 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066402400000000004 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599812+0000", 2024-09-14T07:00:51.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7689999999999998e-06 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175549 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594075+0000", 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41868786000000002, 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42441252000000002, 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.867 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175549 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594075+0000", 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594075+0000", 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594075+0000", 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8000000000000001e-07 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594077+0000", 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594082+0000", 2024-09-14T07:00:51.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6519999999999997e-06 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594086+0000", 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2250000000000002e-06 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600900+0000", 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068138169999999998 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600904+0000", 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7400000000000002e-06 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177702 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596599+0000", 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41616365599999999, 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.421915182, 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.871 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177702 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596599+0000", 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596599+0000", 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596599+0000", 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2000000000000001e-07 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596601+0000", 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5099999999999999e-06 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596605+0000", 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2849999999999996e-06 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596608+0000", 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.615e-06 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602097+0000", 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054896160000000001 2024-09-14T07:00:51.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602106+0000", 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7320000000000004e-06 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179632 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.875 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.614984+0000", 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.397778572, 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.403557693, 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179632 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614984+0000", 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614984+0000", 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614985+0000", 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.32e-07 2024-09-14T07:00:51.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614987+0000", 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.153e-06 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614993+0000", 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9839999999999998e-06 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.614997+0000", 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7339999999999999e-06 2024-09-14T07:00:51.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615076+0000", 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8467999999999995e-05 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615084+0000", 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.028e-06 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180581 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615633+0000", 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39712917599999997, 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.402934811, 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180581 2024-09-14T07:00:51.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615633+0000", 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615633+0000", 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615634+0000", 2024-09-14T07:00:51.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9299999999999999e-07 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615638+0000", 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5639999999999998e-06 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615643+0000", 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615647+0000", 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2799999999999999e-06 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615987+0000", 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00033991199999999999 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615991+0000", 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3730000000000003e-06 2024-09-14T07:00:51.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180746 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621413+0000", 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39134938000000002, 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39718247400000001, 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.884 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180746 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621413+0000", 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621413+0000", 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621413+0000", 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1199999999999999e-07 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621415+0000", 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.071e-06 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621418+0000", 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0299999999999998e-06 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621420+0000", 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.627e-06 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621782+0000", 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000361851 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621791+0000", 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7639999999999994e-06 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:181010 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628530+0000", 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38423313199999998, 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39009275399999999, 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 181010 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628530+0000", 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628530+0000", 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628530+0000", 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8999999999999998e-07 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628531+0000", 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.347e-06 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628535+0000", 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8010000000000001e-06 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628538+0000", 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9859999999999999e-06 2024-09-14T07:00:51.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628751+0000", 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000212833 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628755+0000", 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3889999999999998e-06 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172235 3.8 3.e79f5d48 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.850956+0000", 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161806635, 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1676940330000001, 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172235 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850956+0000", 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850956+0000", 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850956+0000", 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.65e-07 2024-09-14T07:00:51.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850957+0000", 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.068e-06 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850960+0000", 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1310000000000002e-06 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850963+0000", 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.576e-06 2024-09-14T07:00:51.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851095+0000", 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00013207099999999999 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851099+0000", 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5379999999999998e-06 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596477+0000", 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74537861299999997 2024-09-14T07:00:51.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596485+0000", 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7000000000000008e-06 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172726 3.8 3.400a7c28 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1612898810000001, 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.167208794, 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.896 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172726 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:51.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851474+0000", 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2139999999999999e-06 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851477+0000", 2024-09-14T07:00:51.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.208e-06 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851480+0000", 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4700000000000001e-06 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852248+0000", 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00076849899999999996 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852257+0000", 2024-09-14T07:00:51.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6840000000000002e-06 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597699+0000", 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74544160400000004 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597706+0000", 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4170000000000003e-06 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173390 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592480+0000", 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.420282934, 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42623274500000002, 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173390 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592480+0000", 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592480+0000", 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592480+0000", 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9999999999999999e-07 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592481+0000", 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0419999999999999e-06 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592485+0000", 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8430000000000003e-06 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592487+0000", 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T07:00:51.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598732+0000", 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0062443439999999998 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598740+0000", 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2889999999999998e-06 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174905 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593174+0000", 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41958854299999998, 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42556595000000003, 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174905 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.905 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593174+0000", 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593174+0000", 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593174+0000", 2024-09-14T07:00:51.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593177+0000", 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1569999999999998e-06 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593183+0000", 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5950000000000004e-06 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593188+0000", 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7859999999999996e-06 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599842+0000", 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066541500000000002 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599850+0000", 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8469999999999996e-06 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175550 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594092+0000", 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41867079099999999, 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42467590199999999, 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175550 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.909 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594092+0000", 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594092+0000", 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594092+0000", 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0899999999999997e-07 2024-09-14T07:00:51.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594094+0000", 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.841e-06 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594099+0000", 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.617e-06 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594104+0000", 2024-09-14T07:00:51.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6579999999999999e-06 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600937+0000", 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068327259999999999 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600945+0000", 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7389999999999999e-06 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177703 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596611+0000", 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41615150699999998, 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42218552799999998, 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177703 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596611+0000", 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596611+0000", 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596611+0000", 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2000000000000001e-07 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596612+0000", 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6899999999999996e-07 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596615+0000", 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0680000000000001e-06 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596618+0000", 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3609999999999999e-06 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602130+0000", 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0055123009999999998 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602134+0000", 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6849999999999999e-06 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179636 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.916 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615015+0000", 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39774763299999999, 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40380956499999998, 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179636 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615015+0000", 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615015+0000", 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615015+0000", 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.96e-07 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615017+0000", 2024-09-14T07:00:51.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9360000000000002e-06 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615024+0000", 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2090000000000002e-06 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615028+0000", 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615126+0000", 2024-09-14T07:00:51.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8566000000000005e-05 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615134+0000", 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8970000000000008e-06 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180583 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615651+0000", 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.397112152, 2024-09-14T07:00:51.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.403200222, 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180583 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615651+0000", 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615651+0000", 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620802+0000", 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051518060000000001 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620808+0000", 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1639999999999998e-06 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620817+0000", 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3599999999999996e-06 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620822+0000", 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3249999999999998e-06 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620877+0000", 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5316999999999999e-05 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620886+0000", 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6770000000000001e-06 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180748 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621435+0000", 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.391327488, 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39744307600000001, 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.924 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180748 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621435+0000", 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621435+0000", 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621435+0000", 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1600000000000002e-07 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621437+0000", 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8700000000000001e-06 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621443+0000", 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8039999999999998e-06 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621448+0000", 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.13e-06 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621818+0000", 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000369391 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621824+0000", 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7399999999999998e-06 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:181011 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.628543+0000", 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.38422012799999999, 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39036205699999998, 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.928 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 181011 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628543+0000", 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628543+0000", 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628543+0000", 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.05e-07 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628544+0000", 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1909999999999999e-06 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628548+0000", 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0990000000000001e-06 2024-09-14T07:00:51.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628552+0000", 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2289999999999997e-06 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628770+0000", 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000217586 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.628774+0000", 2024-09-14T07:00:51.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9049999999999999e-06 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172257 3.8 3.304f6de8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.850979+0000", 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1617837879999999, 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1679535940000001, 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.932 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172257 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850979+0000", 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850979+0000", 2024-09-14T07:00:51.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850979+0000", 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7000000000000001e-07 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850981+0000", 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.373e-06 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850984+0000", 2024-09-14T07:00:51.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7639999999999999e-06 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850987+0000", 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.57e-06 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851131+0000", 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014424500000000001 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851138+0000", 2024-09-14T07:00:51.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3150000000000003e-06 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596519+0000", 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74538055599999997 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596527+0000", 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5889999999999997e-06 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172760 3.8 3.a1afb768 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161266855, 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.16746846, 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172760 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.937 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:51.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851497+0000", 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1209999999999999e-06 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851501+0000", 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3279999999999999e-06 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851505+0000", 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.275e-06 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852297+0000", 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079227500000000005 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852306+0000", 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6659999999999997e-06 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597736+0000", 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74543018500000002 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597743+0000", 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4290000000000004e-06 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173391 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592501+0000", 2024-09-14T07:00:51.941 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42026150000000001, 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42649450999999999, 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173391 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592501+0000", 2024-09-14T07:00:51.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592501+0000", 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592502+0000", 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4700000000000002e-07 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592503+0000", 2024-09-14T07:00:51.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.403e-06 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592507+0000", 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.765e-06 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592509+0000", 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6819999999999999e-06 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598764+0000", 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0062543049999999999 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598767+0000", 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6349999999999999e-06 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174907 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593209+0000", 2024-09-14T07:00:51.945 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41955410500000001, 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42581513300000001, 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174907 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593209+0000", 2024-09-14T07:00:51.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593209+0000", 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593209+0000", 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9800000000000003e-07 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593219+0000", 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5519999999999993e-06 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593225+0000", 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2419999999999997e-06 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593230+0000", 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8670000000000002e-06 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599876+0000", 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066458369999999999 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599879+0000", 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.692e-06 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175551 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.949 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594110+0000", 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41865243099999999, 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42494198, 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175551 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594110+0000", 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594110+0000", 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594111+0000", 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5899999999999996e-07 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594113+0000", 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.86e-06 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594119+0000", 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5610000000000004e-06 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594125+0000", 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7679999999999997e-06 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600970+0000", 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068446150000000001 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600974+0000", 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3109999999999999e-06 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.953 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:177704 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.596637+0000", 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41612536100000003, 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42244141299999999, 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 177704 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596637+0000", 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596637+0000", 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596638+0000", 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7399999999999999e-07 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596640+0000", 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.854e-06 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596645+0000", 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8359999999999997e-06 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596652+0000", 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0480000000000001e-06 2024-09-14T07:00:51.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602157+0000", 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0055047000000000004 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.602165+0000", 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4699999999999996e-06 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.957 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179868 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615032+0000", 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39773051399999998, 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40407409399999999, 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179868 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615032+0000", 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615032+0000", 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615032+0000", 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9499999999999998e-07 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615038+0000", 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4439999999999999e-06 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615043+0000", 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7169999999999999e-06 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615047+0000", 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.053e-06 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615170+0000", 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000123399 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615180+0000", 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0049e-05 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180585 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.620839+0000", 2024-09-14T07:00:51.961 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.391923831, 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39829431700000001, 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180585 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620839+0000", 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620839+0000", 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620839+0000", 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620841+0000", 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1490000000000001e-06 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620847+0000", 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0799999999999996e-06 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620850+0000", 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0960000000000001e-06 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620925+0000", 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5754999999999996e-05 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620935+0000", 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.55e-06 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180750 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.965 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621454+0000", 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39130837899999998, 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39770635900000001, 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180750 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621454+0000", 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621454+0000", 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621455+0000", 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4199999999999995e-07 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.967 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621457+0000", 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.697e-06 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621462+0000", 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8560000000000003e-06 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621467+0000", 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4769999999999997e-06 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621852+0000", 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038458200000000001 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621856+0000", 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1189999999999999e-06 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.972 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172317 3.8 3.ec3f5dc8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.972 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.850991+0000", 2024-09-14T07:00:51.972 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1617719959999999, 2024-09-14T07:00:51.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1681969759999999, 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172317 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850991+0000", 2024-09-14T07:00:51.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850991+0000", 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850991+0000", 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850992+0000", 2024-09-14T07:00:51.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.209e-06 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850995+0000", 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0649999999999999e-06 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.850998+0000", 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1099999999999999e-06 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851163+0000", 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000165153 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851168+0000", 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.454e-06 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596560+0000", 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745392469 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596573+0000", 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2677000000000001e-05 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172775 3.8 3.c94e7208 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161238784, 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1678624799999999, 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.977 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172775 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6100000000000002e-07 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851531+0000", 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2760000000000003e-06 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851536+0000", 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2640000000000002e-06 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851538+0000", 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5289999999999998e-06 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852358+0000", 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081989599999999997 2024-09-14T07:00:51.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852368+0000", 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597767+0000", 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74539930200000004 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597771+0000", 2024-09-14T07:00:51.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4039999999999997e-06 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173946 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592514+0000", 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42024881200000003, 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42690615399999998, 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.982 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173946 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592514+0000", 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592514+0000", 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592514+0000", 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2000000000000001e-07 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592515+0000", 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.065e-06 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592519+0000", 2024-09-14T07:00:51.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6519999999999999e-06 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592522+0000", 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6539999999999999e-06 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598802+0000", 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0062808910000000003 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598811+0000", 2024-09-14T07:00:51.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7120000000000006e-06 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174908 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593258+0000", 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.419504824, 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42618887900000002, 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.986 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174908 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593258+0000", 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593258+0000", 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593258+0000", 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6000000000000003e-07 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593261+0000", 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2749999999999998e-06 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593272+0000", 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1518000000000001e-05 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593284+0000", 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1333e-05 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599908+0000", 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066245890000000002 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599916+0000", 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4699999999999996e-06 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.989 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175552 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594131+0000", 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41863209099999998, 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42534380500000002, 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175552 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594131+0000", 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594131+0000", 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594131+0000", 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5699999999999998e-07 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594133+0000", 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7910000000000001e-06 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594140+0000", 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7889999999999997e-06 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594145+0000", 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0669999999999999e-06 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601003+0000", 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068585529999999999 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601011+0000", 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5549999999999997e-06 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179869 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.993 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615052+0000", 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39771102800000002, 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.404450386, 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179869 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615052+0000", 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615052+0000", 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615052+0000", 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1699999999999999e-07 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615054+0000", 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615060+0000", 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5010000000000001e-06 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615065+0000", 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4309999999999996e-06 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615217+0000", 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000152239 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615226+0000", 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5270000000000004e-06 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180587 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.620867+0000", 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39189515800000002, 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39866124400000003, 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:51.997 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180587 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620867+0000", 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620867+0000", 2024-09-14T07:00:51.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620868+0000", 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620870+0000", 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2350000000000002e-06 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620876+0000", 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1539999999999999e-06 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:51.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620881+0000", 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5140000000000003e-06 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620976+0000", 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5612000000000005e-05 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620985+0000", 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6100000000000006e-06 2024-09-14T07:00:52.000 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180752 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621491+0000", 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39127168400000001, 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39806637900000003, 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.001 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180752 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621491+0000", 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621491+0000", 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.002 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621491+0000", 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1199999999999998e-07 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621496+0000", 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6739999999999996e-06 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621500+0000", 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2180000000000001e-06 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621503+0000", 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7310000000000002e-06 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621887+0000", 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038370800000000002 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621896+0000", 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9919999999999996e-06 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.004 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172332 3.8 3.fbfc8308 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851018+0000", 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161744307, 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.168565209, 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172332 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.005 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851018+0000", 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851018+0000", 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851019+0000", 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8900000000000001e-07 2024-09-14T07:00:52.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851020+0000", 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4699999999999999e-06 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851024+0000", 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7060000000000002e-06 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851027+0000", 2024-09-14T07:00:52.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851198+0000", 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000170357 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851205+0000", 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1919999999999999e-06 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596607+0000", 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74540262599999996 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596615+0000", 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.447e-06 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172778 3.8 3.d3d5f748 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:52.009 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1612206730000001, 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1680738070000001, 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172778 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:52.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3900000000000001e-07 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851543+0000", 2024-09-14T07:00:52.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851546+0000", 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851549+0000", 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4530000000000001e-06 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852399+0000", 2024-09-14T07:00:52.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00085052200000000004 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852407+0000", 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.238e-06 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597814+0000", 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74540710700000001 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597821+0000", 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173948 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592525+0000", 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.420238055, 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42712341300000001, 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.014 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173948 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592525+0000", 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592525+0000", 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592525+0000", 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592526+0000", 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0559999999999999e-06 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592529+0000", 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.106e-06 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592532+0000", 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.565e-06 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598837+0000", 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0063052660000000003 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598845+0000", 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5140000000000001e-06 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.017 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174909 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593299+0000", 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.419463698, 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.426375791, 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174909 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.018 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593299+0000", 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593299+0000", 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593300+0000", 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2799999999999996e-07 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593302+0000", 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2419999999999999e-06 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593308+0000", 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593314+0000", 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2859999999999999e-06 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599943+0000", 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006629907 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599947+0000", 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.89e-06 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175553 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594152+0000", 2024-09-14T07:00:52.021 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.418610914, 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.425550663, 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175553 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594152+0000", 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594152+0000", 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594152+0000", 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6300000000000005e-07 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594155+0000", 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2089999999999999e-06 2024-09-14T07:00:52.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594165+0000", 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0377000000000001e-05 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594173+0000", 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3140000000000004e-06 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601036+0000", 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006862997 2024-09-14T07:00:52.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601040+0000", 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:178878 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.609122+0000", 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40364079400000002, 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41060694599999997, 2024-09-14T07:00:52.025 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 178878 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609122+0000", 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609122+0000", 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609122+0000", 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2499999999999999e-07 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609126+0000", 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3340000000000003e-06 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609131+0000", 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3710000000000002e-06 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609133+0000", 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.452e-06 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609173+0000", 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0062999999999999e-05 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609182+0000", 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7159999999999992e-06 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179870 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615071+0000", 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39769211399999999, 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40468599399999999, 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.029 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179870 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615071+0000", 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615071+0000", 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615071+0000", 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.46e-07 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615073+0000", 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.699e-06 2024-09-14T07:00:52.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615078+0000", 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7490000000000004e-06 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615083+0000", 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6179999999999997e-06 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615259+0000", 2024-09-14T07:00:52.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017599799999999999 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615267+0000", 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7919999999999993e-06 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180588 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.620886+0000", 2024-09-14T07:00:52.033 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39187709100000001, 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.398901218, 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180588 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620886+0000", 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620886+0000", 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620886+0000", 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3999999999999998e-07 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620887+0000", 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4020000000000001e-06 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620891+0000", 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1099999999999996e-06 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620896+0000", 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1560000000000003e-06 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621019+0000", 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000122609 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621027+0000", 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5989999999999996e-06 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180754 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621507+0000", 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39125590900000001, 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39830944099999999, 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.037 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180754 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621507+0000", 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621507+0000", 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621507+0000", 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2499999999999999e-07 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621508+0000", 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0529999999999999e-06 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621511+0000", 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1159999999999999e-06 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621514+0000", 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4329999999999998e-06 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621944+0000", 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043068499999999999 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621952+0000", 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1769999999999998e-06 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.040 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172349 3.8 3.f4a00da8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851038+0000", 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1617245039999999, 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.168805638, 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172349 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.041 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851038+0000", 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851038+0000", 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851038+0000", 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8999999999999998e-07 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851040+0000", 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.477e-06 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851045+0000", 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4489999999999998e-06 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851050+0000", 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2420000000000002e-06 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851229+0000", 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017901399999999999 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851232+0000", 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.461e-06 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596645+0000", 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74541288100000003 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596652+0000", 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4599999999999997e-06 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173950 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592536+0000", 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.420226345, 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42733791799999998, 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.045 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173950 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592536+0000", 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592536+0000", 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592537+0000", 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592538+0000", 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0070000000000001e-06 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592541+0000", 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2459999999999999e-06 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592544+0000", 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1049999999999999e-06 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598876+0000", 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0063321209999999996 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598884+0000", 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3159999999999997e-06 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.048 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174910 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593333+0000", 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41942972000000001, 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42656856700000001, 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174910 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.049 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593333+0000", 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593333+0000", 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593334+0000", 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1200000000000003e-07 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593336+0000", 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.52e-06 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593345+0000", 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5869999999999993e-06 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593349+0000", 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7160000000000002e-06 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599978+0000", 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.006628452 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.599985+0000", 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4390000000000003e-06 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175554 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.052 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594183+0000", 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41857968200000001, 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.425744872, 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175554 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.053 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594183+0000", 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594183+0000", 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594183+0000", 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3700000000000001e-07 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594185+0000", 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.03e-06 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594191+0000", 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0140000000000004e-06 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594196+0000", 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5480000000000003e-06 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601069+0000", 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068733479999999996 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601077+0000", 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1519999999999992e-06 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.056 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:178879 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.609139+0000", 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40362336100000001, 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.410816287, 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 178879 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.057 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609139+0000", 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609139+0000", 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609140+0000", 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7300000000000002e-07 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609141+0000", 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.705e-06 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609147+0000", 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5339999999999999e-06 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609150+0000", 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9349999999999999e-06 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609221+0000", 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0712999999999999e-05 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609231+0000", 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0614999999999999e-05 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179888 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.060 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615089+0000", 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.397673942, 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40489429399999999, 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179888 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.061 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615089+0000", 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615089+0000", 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615089+0000", 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6300000000000001e-07 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.062 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615091+0000", 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7430000000000001e-06 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615097+0000", 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7309999999999998e-06 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615102+0000", 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2650000000000004e-06 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615301+0000", 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019880399999999999 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615308+0000", 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5059999999999998e-06 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180589 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.064 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.620901+0000", 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39186128799999997, 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.399108085, 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180589 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620901+0000", 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620901+0000", 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620902+0000", 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2600000000000002e-07 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620904+0000", 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.875e-06 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620910+0000", 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.767e-06 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620914+0000", 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6789999999999996e-06 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621063+0000", 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000148558 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621070+0000", 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.61e-06 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180756 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.068 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621516+0000", 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39124621799999998, 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.398521083, 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180756 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621516+0000", 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621516+0000", 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621517+0000", 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.16e-07 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621518+0000", 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6099999999999999e-07 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621521+0000", 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6090000000000001e-06 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621524+0000", 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5409999999999999e-06 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621983+0000", 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00045940400000000001 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621988+0000", 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9130000000000002e-06 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.072 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172350 3.8 3.f169228 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851053+0000", 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.1617093759999999, 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.169010903, 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172350 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.073 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851053+0000", 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851053+0000", 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851053+0000", 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851055+0000", 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0440000000000001e-06 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851058+0000", 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.287e-06 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851060+0000", 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4569999999999999e-06 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851263+0000", 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020244100000000001 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851270+0000", 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1600000000000001e-06 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596688+0000", 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74541829900000001 2024-09-14T07:00:52.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596696+0000", 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0050000000000004e-06 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173952 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592547+0000", 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42021576999999999, 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42754942099999999, 2024-09-14T07:00:52.077 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173952 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592547+0000", 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592547+0000", 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592547+0000", 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8999999999999998e-07 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592548+0000", 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0330000000000001e-06 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592551+0000", 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1709999999999998e-06 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592555+0000", 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.315e-06 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598909+0000", 2024-09-14T07:00:52.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0063538689999999998 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598913+0000", 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9589999999999997e-06 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174911 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593357+0000", 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41940579099999997, 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42676695999999997, 2024-09-14T07:00:52.081 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174911 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593357+0000", 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593357+0000", 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593357+0000", 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593359+0000", 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9039999999999999e-06 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593366+0000", 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1679999999999998e-06 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593371+0000", 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3120000000000004e-06 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600010+0000", 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066391669999999996 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600013+0000", 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6289999999999999e-06 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175555 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594221+0000", 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.418541628, 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42593165300000002, 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.085 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175555 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594221+0000", 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594221+0000", 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594222+0000", 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7100000000000002e-07 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594228+0000", 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3810000000000004e-06 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594235+0000", 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1590000000000004e-06 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594246+0000", 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0890000000000001e-05 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601101+0000", 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068549680000000003 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601105+0000", 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.675e-06 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:178880 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.609168+0000", 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40359451600000001, 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41101415699999999, 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 178880 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609168+0000", 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609168+0000", 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609169+0000", 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8699999999999995e-07 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609170+0000", 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7549999999999999e-06 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609177+0000", 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0539999999999997e-06 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609181+0000", 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7160000000000001e-06 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609269+0000", 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7787999999999994e-05 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609277+0000", 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1340000000000004e-06 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179889 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.092 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615117+0000", 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.397645902, 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.405091217, 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179889 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615117+0000", 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615117+0000", 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615117+0000", 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7799999999999997e-07 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.094 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615118+0000", 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2979999999999999e-06 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615122+0000", 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6550000000000002e-06 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615125+0000", 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6120000000000001e-06 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.095 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615342+0000", 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000217402 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615352+0000", 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0043e-05 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180590 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.620920+0000", 2024-09-14T07:00:52.096 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39184254200000002, 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39931554200000002, 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180590 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620920+0000", 2024-09-14T07:00:52.097 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620920+0000", 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620921+0000", 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620922+0000", 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8109999999999999e-06 2024-09-14T07:00:52.098 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620928+0000", 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5779999999999998e-06 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620933+0000", 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7400000000000004e-06 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621109+0000", 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00017631799999999999 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.099 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621117+0000", 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.259e-06 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180758 3.8 3.27b983c8 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621527+0000", 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39123592200000001, 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39873749000000003, 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.100 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180758 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621527+0000", 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.101 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621527+0000", 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621527+0000", 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.17e-07 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621528+0000", 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1119999999999999e-06 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.102 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621534+0000", 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621537+0000", 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.439e-06 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.622018+0000", 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048099299999999998 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.103 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.622022+0000", 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8330000000000004e-06 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172370 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851063+0000", 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 1.161699313, 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.169227561, 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.104 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172370 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851063+0000", 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851063+0000", 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.105 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851064+0000", 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851065+0000", 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1790000000000001e-06 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851068+0000", 2024-09-14T07:00:52.106 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9730000000000002e-06 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851070+0000", 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4490000000000002e-06 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851294+0000", 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00022417699999999999 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.107 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851298+0000", 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4199999999999999e-06 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596731+0000", 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74543332699999998 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.596739+0000", 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8979999999999996e-06 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.108 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:173954 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.592564+0000", 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.42019899799999999, 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42775907000000002, 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 173954 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.109 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592564+0000", 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592564+0000", 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592564+0000", 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1500000000000003e-07 2024-09-14T07:00:52.110 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592566+0000", 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4449999999999999e-06 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592570+0000", 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.895e-06 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592573+0000", 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.112e-06 2024-09-14T07:00:52.111 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598942+0000", 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0063694379999999998 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.598951+0000", 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.191e-06 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.112 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:174913 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.593377+0000", 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41938588799999998, 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42697311100000002, 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 174913 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.113 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593377+0000", 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593377+0000", 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593377+0000", 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.72e-07 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.114 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593379+0000", 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9860000000000001e-06 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593385+0000", 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8080000000000001e-06 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.593390+0000", 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7090000000000001e-06 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.115 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600043+0000", 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0066532120000000004 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.600051+0000", 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6890000000000004e-06 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:175556 3.8 3.134c5a8 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.116 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.594269+0000", 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.41849386399999999, 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.42610852700000001, 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 175556 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.117 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594269+0000", 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594269+0000", 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594269+0000", 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.3799999999999997e-07 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.118 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594272+0000", 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1900000000000002e-06 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594278+0000", 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.263e-06 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.594287+0000", 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3239999999999992e-06 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.119 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601134+0000", 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0068464279999999999 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.601141+0000", 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5979999999999999e-06 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:178881 3.8 3.ad9ae408 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.120 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.609186+0000", 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.40357707199999998, 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.41121940000000001, 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 178881 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609186+0000", 2024-09-14T07:00:52.121 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609186+0000", 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609186+0000", 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.51e-07 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.122 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609188+0000", 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7689999999999999e-06 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609193+0000", 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8810000000000004e-06 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609196+0000", 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.123 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609317+0000", 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000121543 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.609325+0000", 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9959999999999993e-06 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:179890 3.8 3.2a7aed08 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.615128+0000", 2024-09-14T07:00:52.124 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39763453500000001, 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.40530347300000003, 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 179890 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615128+0000", 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.125 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615128+0000", 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615128+0000", 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3999999999999998e-07 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615129+0000", 2024-09-14T07:00:52.126 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.043e-06 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615133+0000", 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8290000000000001e-06 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615142+0000", 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9630000000000004e-06 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.127 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615387+0000", 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000244701 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.615394+0000", 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4850000000000003e-06 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180591 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.620939+0000", 2024-09-14T07:00:52.128 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39182327099999997, 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39951987300000003, 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180591 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620939+0000", 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.129 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620939+0000", 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620940+0000", 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1100000000000001e-07 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620942+0000", 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.733e-06 2024-09-14T07:00:52.130 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620947+0000", 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7139999999999998e-06 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.620953+0000", 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5809999999999996e-06 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621153+0000", 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00020015399999999999 2024-09-14T07:00:52.131 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621161+0000", 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4740000000000005e-06 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:180765 3.8 3.fa5c7d68 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:50.621551+0000", 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 0.39121141399999998, 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.39893471699999999, 2024-09-14T07:00:52.132 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "waiting for peered", 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 180765 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621551+0000", 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.133 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621551+0000", 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621552+0000", 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9299999999999999e-07 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621553+0000", 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.815e-06 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.134 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621559+0000", 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7819999999999999e-06 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.621564+0000", 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.622044+0000", 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000480106 2024-09-14T07:00:52.135 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.622048+0000", 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8700000000000002e-06 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: ], 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 256 2024-09-14T07:00:52.136 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:52.137 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:52.138 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:52.138 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:52.139 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:52.139 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:52.137+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:52.139 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:52.140 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:52.140 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:52.140 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:52.140 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:52.141 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T07:00:52.239 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:52.238+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:52.275 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:52.329 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:52.329 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:52.339 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:52.339+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:52.342 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:52.440 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:52.439+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:52.495 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:52.495 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:52.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:52.542 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:52.541+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:52.621 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:52.621 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:52.624 INFO:teuthology.orchestra.run.smithi007.stdout:5882 write 0x8d0f15e thru 0x8d18ad2 (0x9975 bytes) 2024-09-14T07:00:52.628 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:52.628 INFO:teuthology.orchestra.run.smithi007.stdout:5891 punch from 0x4b0bbc4 to 0x4b17ccc, (0xc108 bytes) 2024-09-14T07:00:52.629 INFO:teuthology.orchestra.run.smithi007.stdout:5893 read 0x58de415 thru 0x58e570b (0x72f7 bytes) 2024-09-14T07:00:52.629 INFO:teuthology.orchestra.run.smithi007.stdout:5894 write 0xd7cc36 thru 0xd7d9cd (0xd98 bytes) 2024-09-14T07:00:52.631 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:52.633 INFO:teuthology.orchestra.run.smithi007.stdout:5895 writesame 0xb20aaa thru 0xb22511 data_size 0x34(0x1a68 bytes) 2024-09-14T07:00:52.640 INFO:teuthology.orchestra.run.smithi007.stdout:5896 punch from 0x30a53ff to 0x30ae36a, (0x8f6b bytes) 2024-09-14T07:00:52.642 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:52.641+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:52.642 INFO:teuthology.orchestra.run.smithi007.stdout:5897 trunc from 0x8d18ad3 to 0xae55cc9 2024-09-14T07:00:52.647 INFO:teuthology.orchestra.run.smithi007.stdout:5898 read 0x6235d1 thru 0x62c316 (0x8d46 bytes) 2024-09-14T07:00:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:52.648 INFO:teuthology.orchestra.run.smithi007.stdout:5902 read 0x1e7a735 thru 0x1e832c5 (0x8b91 bytes) 2024-09-14T07:00:52.649 INFO:teuthology.orchestra.run.smithi007.stdout:5903 punch from 0xaa9956 to 0xab4790, (0xae3a bytes) 2024-09-14T07:00:52.649 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:52.649 INFO:teuthology.orchestra.run.smithi007.stdout:5907 write 0x1588495 thru 0x1589341 (0xead bytes) 2024-09-14T07:00:52.653 INFO:teuthology.orchestra.run.smithi007.stdout:5908 read 0x5ab017d thru 0x5ab930a (0x918e bytes) 2024-09-14T07:00:52.653 INFO:teuthology.orchestra.run.smithi007.stdout:5909 punch from 0xa57720b to 0xa582ecc, (0xbcc1 bytes) 2024-09-14T07:00:52.653 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:52.653 INFO:teuthology.orchestra.run.smithi007.stdout:5911 write 0x2ce650b thru 0x2cedb04 (0x75fa bytes) 2024-09-14T07:00:52.657 INFO:teuthology.orchestra.run.smithi007.stdout:5912 read 0x2364ce thru 0x238a89 (0x25bc bytes) 2024-09-14T07:00:52.657 INFO:teuthology.orchestra.run.smithi007.stdout:5914 read 0x7eba306 thru 0x7ec915b (0xee56 bytes) 2024-09-14T07:00:52.663 INFO:teuthology.orchestra.run.smithi007.stdout:5918 writesame 0xb66bb1c thru 0xb67a24c data_size 0x23(0xe731 bytes) 2024-09-14T07:00:52.664 INFO:teuthology.orchestra.run.smithi007.stdout:5919 read 0x88c08a0 thru 0x88cb034 (0xa795 bytes) 2024-09-14T07:00:52.664 INFO:teuthology.orchestra.run.smithi007.stdout:5920 punch from 0x58bdec3 to 0x58c7d3d, (0x9e7a bytes) 2024-09-14T07:00:52.664 INFO:teuthology.orchestra.run.smithi007.stdout:5921 read 0x4d5734f thru 0x4d649db (0xd68d bytes) 2024-09-14T07:00:52.664 INFO:teuthology.orchestra.run.smithi007.stdout:5922 write 0x688d7f8 thru 0x6890b07 (0x3310 bytes) 2024-09-14T07:00:52.668 INFO:teuthology.orchestra.run.smithi007.stdout:5923 read 0x15d41d1 thru 0x15d4868 (0x698 bytes) 2024-09-14T07:00:52.668 INFO:teuthology.orchestra.run.smithi007.stdout:5924 read 0xb3298e2 thru 0xb32a2be (0x9dd bytes) 2024-09-14T07:00:52.679 INFO:teuthology.orchestra.run.smithi007.stdout:5925 trunc from 0xb67a24d to 0x1dd0fda 2024-09-14T07:00:52.711 INFO:teuthology.orchestra.run.smithi007.stdout:5926 write 0x30dbbee thru 0x30e01b7 (0x45ca bytes) 2024-09-14T07:00:52.715 INFO:teuthology.orchestra.run.smithi007.stdout:5927 read 0x3000171 thru 0x300606c (0x5efc bytes) 2024-09-14T07:00:52.733 INFO:teuthology.orchestra.run.smithi007.stdout:5928 write 0xd3a573d thru 0xd3a7f6b (0x282f bytes) 2024-09-14T07:00:52.737 INFO:teuthology.orchestra.run.smithi007.stdout:5929 read 0xc9770a5 thru 0xc97dd3e (0x6c9a bytes) 2024-09-14T07:00:52.737 INFO:teuthology.orchestra.run.smithi007.stdout:5930 read 0x1edf143 thru 0x1ee3f59 (0x4e17 bytes) 2024-09-14T07:00:52.737 INFO:teuthology.orchestra.run.smithi007.stdout:5931 write 0x4412f7 thru 0x44b540 (0xa24a bytes) 2024-09-14T07:00:52.742 INFO:teuthology.orchestra.run.smithi007.stdout:5932 read 0xc2ca79c thru 0xc2d1384 (0x6be9 bytes) 2024-09-14T07:00:52.743 INFO:teuthology.orchestra.run.smithi007.stdout:5933 read 0xc12b8cd thru 0xc1398bf (0xdff3 bytes) 2024-09-14T07:00:52.743 INFO:teuthology.orchestra.run.smithi007.stdout:5934 punch from 0x82f932d to 0x8308eca, (0xfb9d bytes) 2024-09-14T07:00:52.743 INFO:teuthology.orchestra.run.smithi007.stdout:5936 read 0xcb83a37 thru 0xcb87439 (0x3a03 bytes) 2024-09-14T07:00:52.743 INFO:teuthology.orchestra.run.smithi007.stdout:5937 clone 34 order 23 su 32768 sc 7 2024-09-14T07:00:52.744 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:52.742+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:52.750 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:52.750 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:52.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:52.844 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:52.843+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:52.907 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:52.907 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:52.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:52.944 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:52.943+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:53.022 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:53.023 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:53.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:53.045 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:53.044+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:53.138 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:53.138 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:53.145 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:53.145+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:53.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:53.247 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:53.247+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:53.253 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:53.254 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:53.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:53.348 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:53.347+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:53.369 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:53.369 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:53.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:53.449 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:53.448+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:53.485 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:53.485 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:53.493 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:53.549 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:53.549+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:53.648 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:53.648 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:53.651 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:53.650+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:53.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:53.663 INFO:teuthology.orchestra.run.smithi007.stdout:flattening image image_client.0-clone33 2024-09-14T07:00:53.752 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:53.751+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:53.802 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:53.802 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:53.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:53.853 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:53.853+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:53.892 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-14T07:00:53.943 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:53.943 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:53.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:53.954 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:53.953+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:53.993 INFO:teuthology.orchestra.run.smithi007.stdout:5939 punch from 0xa1c6848 to 0xa1cc6b3, (0x5e6b bytes) 2024-09-14T07:00:53.996 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:53.996 INFO:teuthology.orchestra.run.smithi007.stdout:5943 read 0xc9a47d9 thru 0xc9b47a8 (0xffd0 bytes) 2024-09-14T07:00:53.996 INFO:teuthology.orchestra.run.smithi007.stdout:5944 write 0xb33e0b1 thru 0xb34cb8b (0xeadb bytes) 2024-09-14T07:00:54.002 INFO:teuthology.orchestra.run.smithi007.stdout:5946 writesame 0x547d85 thru 0x5556e5 data_size 0x1(0xd961 bytes) 2024-09-14T07:00:54.039 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:54.040 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:54.048 INFO:teuthology.orchestra.run.smithi007.stdout:5950 trunc from 0xd3a7f6c to 0x575c01d 2024-09-14T07:00:54.054 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:54.054+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:54.068 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:54.068 INFO:teuthology.orchestra.run.smithi007.stdout:5954 read 0x1710f7b thru 0x1719ff5 (0x907b bytes) 2024-09-14T07:00:54.076 INFO:teuthology.orchestra.run.smithi007.stdout:5955 write 0x74d321b thru 0x74d8f80 (0x5d66 bytes) 2024-09-14T07:00:54.084 INFO:teuthology.orchestra.run.smithi007.stdout:5956 trunc from 0x74d8f81 to 0x2dc507f 2024-09-14T07:00:54.085 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:54.085 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:54.099 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:54.115 INFO:teuthology.orchestra.run.smithi007.stdout:5958 trunc from 0x2dc507f to 0x227f6a0 2024-09-14T07:00:54.139 INFO:teuthology.orchestra.run.smithi007.stdout:5959 writesame 0xc28d6e1 thru 0xc29a621 data_size 0x1(0xcf41 bytes) 2024-09-14T07:00:54.139 INFO:teuthology.orchestra.run.smithi007.stdout:5960 read 0xae98e12 thru 0xaea3d40 (0xaf2f bytes) 2024-09-14T07:00:54.139 INFO:teuthology.orchestra.run.smithi007.stdout:5961 trunc from 0xc29a622 to 0xc021662 2024-09-14T07:00:54.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:00:54.143 INFO:teuthology.orchestra.run.smithi007.stdout:5963 writesame 0x54748a thru 0x55288a data_size 0x7(0xb401 bytes) 2024-09-14T07:00:54.147 INFO:teuthology.orchestra.run.smithi007.stdout:5964 read 0x93a9bc2 thru 0x93b686e (0xccad bytes) 2024-09-14T07:00:54.148 INFO:teuthology.orchestra.run.smithi007.stdout:5965 trunc from 0xc021662 to 0xbe79992 2024-09-14T07:00:54.155 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:54.154+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:54.157 INFO:teuthology.orchestra.run.smithi007.stdout:5966 trunc from 0xbe79992 to 0xeb624c5 2024-09-14T07:00:54.161 INFO:teuthology.orchestra.run.smithi007.stdout:5967 read 0x246867 thru 0x24d350 (0x6aea bytes) 2024-09-14T07:00:54.176 INFO:teuthology.orchestra.run.smithi007.stdout:5968 trunc from 0xeb624c5 to 0x2169980 2024-09-14T07:00:54.184 INFO:teuthology.orchestra.run.smithi007.stdout:5969 punch from 0x14c00ff to 0x14c237d, (0x227e bytes) 2024-09-14T07:00:54.184 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:54.184 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:54.191 INFO:teuthology.orchestra.run.smithi007.stdout:5973 write 0x36cee22 thru 0x36d0588 (0x1767 bytes) 2024-09-14T07:00:54.192 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:54.192 INFO:teuthology.orchestra.run.smithi007.stdout:5976 read 0x2d157ec thru 0x2d20c85 (0xb49a bytes) 2024-09-14T07:00:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:5977 punch from 0x1a60e4c to 0x1a6ed61, (0xdf15 bytes) 2024-09-14T07:00:54.193 INFO:teuthology.orchestra.run.smithi007.stdout:5978 read 0xe76548 thru 0xe78736 (0x21ef bytes) 2024-09-14T07:00:54.206 INFO:teuthology.orchestra.run.smithi007.stdout:5979 writesame 0xa4bd809 thru 0xa4c1114 data_size 0x6(0x390c bytes) 2024-09-14T07:00:54.208 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:54.208 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:54.210 INFO:teuthology.orchestra.run.smithi007.stdout:5980 trunc from 0xa4c1115 to 0x655c573 2024-09-14T07:00:54.217 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:54.221 INFO:teuthology.orchestra.run.smithi007.stdout:5981 writesame 0x67b7256 thru 0x67bf964 data_size 0x19(0x870f bytes) 2024-09-14T07:00:54.221 INFO:teuthology.orchestra.run.smithi007.stdout:5982 punch from 0x5157029 to 0x516071d, (0x96f4 bytes) 2024-09-14T07:00:54.233 INFO:teuthology.orchestra.run.smithi007.stdout:5983 write 0xbc8c454 thru 0xbc96e26 (0xa9d3 bytes) 2024-09-14T07:00:54.237 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:54.242 INFO:teuthology.orchestra.run.smithi007.stdout:5985 trunc from 0xbc96e27 to 0x6a8baba 2024-09-14T07:00:54.252 INFO:teuthology.orchestra.run.smithi007.stdout:5986 read 0x17f7131 thru 0x17ff8f8 (0x87c8 bytes) 2024-09-14T07:00:54.252 INFO:teuthology.orchestra.run.smithi007.stdout:5987 writesame 0x1449a2e thru 0x144e517 data_size 0x2b(0x4aea bytes) 2024-09-14T07:00:54.254 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:54.255 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:54.254+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:54.260 INFO:teuthology.orchestra.run.smithi007.stdout:5989 write 0x6f77710 thru 0x6f86ad5 (0xf3c6 bytes) 2024-09-14T07:00:54.264 INFO:teuthology.orchestra.run.smithi007.stdout:5990 read 0x3f1036c thru 0x3f1a3d2 (0xa067 bytes) 2024-09-14T07:00:54.274 INFO:teuthology.orchestra.run.smithi007.stdout:5992 write 0x9daa487 thru 0x9db2ba3 (0x871d bytes) 2024-09-14T07:00:54.278 INFO:teuthology.orchestra.run.smithi007.stdout:5993 read 0x80ac9e5 thru 0x80b5d84 (0x93a0 bytes) 2024-09-14T07:00:54.279 INFO:teuthology.orchestra.run.smithi007.stdout:compare and write disabled 2024-09-14T07:00:54.279 INFO:teuthology.orchestra.run.smithi007.stdout:5995 punch from 0x587ab44 to 0x587cdda, (0x2296 bytes) 2024-09-14T07:00:54.280 INFO:teuthology.orchestra.run.smithi007.stdout:5999 trunc from 0x9db2ba4 to 0xba5e7d6 2024-09-14T07:00:54.286 INFO:teuthology.orchestra.run.smithi007.stdout:6000 writesame 0x5a2ed7f thru 0x5a3796e data_size 0x8(0x8bf0 bytes) 2024-09-14T07:00:54.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:54.304 INFO:teuthology.orchestra.run.smithi007.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-14T07:00:54.323 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:54.324 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:54.332 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:54.377 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:54.376+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:54.440 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:54.441 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:54.442 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:54.442 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:54.451 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:54.457 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:54.456+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:54.458 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:54.458 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:54.475 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:54.476 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:54.492 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:54.492 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:54.511 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:54.529 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:54.529 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:54.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:54.546 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:54.557 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:54.557+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:54.563 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:54.563 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:54.597 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:54.597 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:54.597 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:54.597 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:54.605 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:54.613 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T07:00:54.690 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:54.689+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:54.699 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:00:54.699 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:00:54.699 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T07:00:54.699 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.592087 2024-09-14T07:00:54.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0.5920869416939005 2024-09-14T07:00:54.751 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:54.752 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:54.752 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:54.752 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:54.752 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:54.759 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:54.758+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:54.760 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:54.860 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:54.860+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:54.906 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:54.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.739567097000005, 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.907 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:00:54.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:00:54.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.088223992000003, 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:00:54.911 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:00:54.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:00:54.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.915 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.088208524000002, 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:00:54.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:00:54.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.088197223000002, 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.920 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081453750000001, 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:00:54.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:00:54.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081441146000003, 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.929 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.933 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081421630000001, 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:00:54.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:00:54.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081409589000003, 2024-09-14T07:00:54.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:00:54.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:00:54.942 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081389287, 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.943 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.947 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081376970999997, 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:00:54.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081365531000003, 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.952 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:00:54.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:00:54.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.956 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081355031999998, 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.957 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:00:54.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.961 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081344839000003, 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:00:54.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081329291999999, 2024-09-14T07:00:54.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:00:54.967 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.967 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.967 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.967 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:00:54.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.967 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:00:54.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:54.971 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081299684000001, 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:00:54.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081278034, 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:00:54.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.081255794999997, 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.980 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.983 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:54.984 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.075659430000002, 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.985 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:00:54.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.988 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 47.075649376999998, 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.989 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:54.993 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:118426 3.b 3:d50c2793:::rbd_data.114f74d20483.000000000000001c:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 42.404267574000002, 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.95135023100000005, 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:54.994 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 118426 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500745+0000", 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500748+0000", 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1329999999999999e-06 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500759+0000", 2024-09-14T07:00:54.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0475000000000001e-05 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500765+0000", 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500824+0000", 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:54.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500830+0000", 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5889999999999999e-06 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450787+0000", 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.94995644099999998 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450901+0000", 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114032 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451052+0000", 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015113 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451746+0000", 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069442799999999997 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452006+0000", 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025975 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:54.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452059+0000", 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3090000000000002e-05 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452095+0000", 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5395999999999998e-05 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.000 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:55.002 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:55.002 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:55.001+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:55.102 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:55.102+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:55.150 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:55.150 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:55.150 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:55.150 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:55.158 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:55.203 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:55.202+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:55.260 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:55.260 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:55.260 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:55.261 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:55.261 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:55.269 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:55.304 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:55.304+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:55.373 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:55.373 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:55.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:55.373 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:55.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 94.607714247000004, 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:00:55.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:00:55.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:00:55.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:00:55.378 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.378 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.378 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 76.534594980999998, 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:00:55.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:00:55.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:00:55.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 76.534338328000004, 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.385 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:00:55.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 76.508051253999994, 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:00:55.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:00:55.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 76.507926639999994, 2024-09-14T07:00:55.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:00:55.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.398 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 76.013307478000002, 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 74.738904817999995, 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:00:55.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:00:55.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.744289207999998, 2024-09-14T07:00:55.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:00:55.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.554116742000005, 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:00:55.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.504983452000005, 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.416 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:00:55.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.420 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.136764487999997, 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:00:55.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.016195549000003, 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:00:55.425 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:00:55.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.885279667999995, 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:00:55.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.856242406000007, 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.434 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:00:55.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.438 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.830472983999996, 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:55.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.797114840999996, 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:00:55.443 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:00:55.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.447 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.632907059999994, 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:00:55.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:00:55.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:00:55.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.621527799999996, 2024-09-14T07:00:55.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:00:55.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.456 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.621480106000007, 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:00:55.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.461 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 70.621137781000002, 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:00:55.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:00:55.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:00:55.466 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.466 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.466 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.466 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.466 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.466 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:55.468 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:55.468 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:55.467+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:55.569 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:55.568+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:55.612 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:55.612 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:55.612 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:55.613 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.0 primary-affinity to 0.592087 (8388032) 2024-09-14T07:00:55.625 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:55.670 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:55.669+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:55.730 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:55.730 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:55.730 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:55.730 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:55.730 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:55.738 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:55.771 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:55.770+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:55.873 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:55.872+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172493 3.8 3:11834377:::rbd_data.114fd4a72a77.0000000000000129:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0310529900000001, 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.395516846, 2024-09-14T07:00:55.884 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172493 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851241+0000", 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164e-06 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851244+0000", 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1259999999999998e-06 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851247+0000", 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851660+0000", 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041335800000000001 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851667+0000", 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9809999999999997e-06 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597160+0000", 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549261700000002 2024-09-14T07:00:55.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597167+0000", 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2629999999999997e-06 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246685+0000", 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649518327 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246756+0000", 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1210000000000004e-05 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172499 3.8 3:1336cd84:::rbd_data.114fd4a72a77.000000000000012f:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0310386830000002, 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3956566389999998, 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.890 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172499 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.22e-07 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851255+0000", 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.114e-06 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851259+0000", 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2909999999999999e-06 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851261+0000", 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.655e-06 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851691+0000", 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043023900000000001 2024-09-14T07:00:55.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851695+0000", 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4000000000000001e-06 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597190+0000", 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745495146 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597194+0000", 2024-09-14T07:00:55.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7050000000000001e-06 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246801+0000", 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649607445 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246911+0000", 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000109422 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172507 3.8 3:17c8295f:::rbd_data.114fd4a72a77.0000000000000136:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0310281310000002, 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3957446349999998, 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172507 2024-09-14T07:00:55.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851265+0000", 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851266+0000", 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851269+0000", 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0079999999999998e-06 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851271+0000", 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.531e-06 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851725+0000", 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00045350300000000001 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851732+0000", 2024-09-14T07:00:55.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9600000000000003e-06 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597223+0000", 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745490926 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597230+0000", 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4499999999999998e-06 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246943+0000", 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649713363 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247009+0000", 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5656999999999998e-05 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172606 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 14336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.901 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0310179230000003, 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.39623221, 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172606 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851276+0000", 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9399999999999993e-07 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851279+0000", 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0070000000000002e-06 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851282+0000", 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9210000000000001e-06 2024-09-14T07:00:55.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851756+0000", 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000474005 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851759+0000", 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3670000000000001e-06 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597256+0000", 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549659599999996 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597260+0000", 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247047+0000", 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649786975 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247140+0000", 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3449999999999995e-05 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247507+0000", 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000366664 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172607 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 40960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.031000637, 2024-09-14T07:00:55.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3965954819999999, 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172607 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:55.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7000000000000001e-07 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851294+0000", 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3710000000000001e-06 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851297+0000", 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7850000000000002e-06 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851309+0000", 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1148000000000001e-05 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851788+0000", 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00047951899999999997 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851803+0000", 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5074e-05 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597290+0000", 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548693700000002 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597298+0000", 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3590000000000008e-06 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247532+0000", 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.650233759 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247590+0000", 2024-09-14T07:00:55.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8091000000000003e-05 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247887+0000", 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029716900000000001 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172608 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 67584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:00:55.913 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0309722920000004, 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3970747800000001, 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172608 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:00:55.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851321+0000", 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0600000000000001e-07 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851322+0000", 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.751e-06 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851331+0000", 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1799999999999996e-06 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851334+0000", 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3409999999999999e-06 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851830+0000", 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00049571000000000001 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851833+0000", 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6069999999999999e-06 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597326+0000", 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549257700000005 2024-09-14T07:00:55.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597333+0000", 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247917+0000", 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6505841430000001 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248020+0000", 2024-09-14T07:00:55.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102581 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248395+0000", 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037526200000000001 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172609 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 94208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:55.919 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0309526460000003, 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3975014290000001, 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172609 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:55.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9200000000000002e-07 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851342+0000", 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.451e-06 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851346+0000", 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9999999999999998e-06 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851348+0000", 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5780000000000001e-06 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851868+0000", 2024-09-14T07:00:55.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051942399999999999 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851876+0000", 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6850000000000007e-06 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597371+0000", 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745494502 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597378+0000", 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6129999999999998e-06 2024-09-14T07:00:55.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248428+0000", 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.651049502 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248482+0000", 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3767999999999997e-05 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248841+0000", 2024-09-14T07:00:55.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00035961400000000002 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172610 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 120832~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0309318420000002, 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3980250019999998, 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.925 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172610 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6800000000000002e-07 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851362+0000", 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4300000000000001e-06 2024-09-14T07:00:55.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851366+0000", 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.811e-06 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851369+0000", 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4739999999999999e-06 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851901+0000", 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053255399999999997 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851905+0000", 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4170000000000001e-06 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597404+0000", 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549902099999998 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597408+0000", 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.527e-06 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248892+0000", 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65148359 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249005+0000", 2024-09-14T07:00:55.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113397 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249386+0000", 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038051300000000002 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172611 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 147456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0309135639999996, 2024-09-14T07:00:55.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3985748930000002, 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172611 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:55.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.79e-07 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851381+0000", 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7379999999999999e-06 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851385+0000", 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5750000000000001e-06 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851387+0000", 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:00:55.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851934+0000", 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00054657299999999996 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851942+0000", 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2549999999999997e-06 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597437+0000", 2024-09-14T07:00:55.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549549800000003 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597445+0000", 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5839999999999997e-06 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249454+0000", 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6520086940000001 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249576+0000", 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012215899999999999 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249954+0000", 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037802300000000001 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172612 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 174080~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.937 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0309019670000001, 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3988136949999999, 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172612 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851392+0000", 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851395+0000", 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2449999999999998e-06 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851398+0000", 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4660000000000002e-06 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851978+0000", 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058031599999999999 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851984+0000", 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1319999999999999e-06 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597469+0000", 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548469799999995 2024-09-14T07:00:55.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597473+0000", 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8510000000000001e-06 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249990+0000", 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6525175509999999 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250091+0000", 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010111900000000001 2024-09-14T07:00:55.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250204+0000", 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113034 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172613 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 200704~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0308919940000001, 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3989707839999999, 2024-09-14T07:00:55.943 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172613 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.05e-07 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851402+0000", 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.026e-06 2024-09-14T07:00:55.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851405+0000", 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2440000000000002e-06 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851407+0000", 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3920000000000001e-06 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852016+0000", 2024-09-14T07:00:55.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060825699999999996 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852024+0000", 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0639999999999994e-06 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597505+0000", 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548078699999998 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597513+0000", 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1659999999999994e-06 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250226+0000", 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652713648 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250279+0000", 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2880999999999997e-05 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250371+0000", 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2114000000000006e-05 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172614 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 227328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0308821840000002, 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3991723779999998, 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.949 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172614 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851411+0000", 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851412+0000", 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4900000000000004e-07 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851415+0000", 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.942e-06 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851417+0000", 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000002e-06 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852054+0000", 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00063717699999999995 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852058+0000", 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6739999999999999e-06 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597543+0000", 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548541400000001 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597548+0000", 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6140000000000002e-06 2024-09-14T07:00:55.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250407+0000", 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652859388 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250487+0000", 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9523000000000004e-05 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250583+0000", 2024-09-14T07:00:55.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6026999999999994e-05 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172615 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 253952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0308725250000004, 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3993917979999999, 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.955 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172615 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851421+0000", 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669999999999999e-06 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851425+0000", 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0110000000000002e-06 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851428+0000", 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.452e-06 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852092+0000", 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000664497 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852099+0000", 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1199999999999996e-06 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597585+0000", 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548563599999995 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597593+0000", 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7989999999999994e-06 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250631+0000", 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653038477 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250689+0000", 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7192999999999999e-05 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250812+0000", 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000123344 2024-09-14T07:00:55.960 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172719 3.8 3:10ac98f7:::rbd_data.114fd4a72a77.000000000000018d:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0308617089999998, 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3995141609999999, 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.961 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172719 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851432+0000", 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851435+0000", 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1980000000000001e-06 2024-09-14T07:00:55.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851442+0000", 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9419999999999998e-06 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852133+0000", 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069105100000000004 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852139+0000", 2024-09-14T07:00:55.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597622+0000", 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548300599999995 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597635+0000", 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3183000000000001e-05 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250847+0000", 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6532118730000001 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250945+0000", 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7770999999999996e-05 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172725 3.8 3:16c5e555:::rbd_data.114fd4a72a77.0000000000000193:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:55.966 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0308350700000002, 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3996608569999998, 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172725 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:55.967 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.968 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851459+0000", 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.626e-06 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851466+0000", 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8349999999999998e-06 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851469+0000", 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:55.969 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852192+0000", 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072263799999999995 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852201+0000", 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2129999999999997e-06 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597665+0000", 2024-09-14T07:00:55.970 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74546438299999995 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597673+0000", 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6410000000000002e-06 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251000+0000", 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653327011 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.971 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251118+0000", 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000118446 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172726 3.8 3:143e5002:::rbd_data.114fd4a72a77.0000000000000194:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0308198150000001, 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3998113490000001, 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.972 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172726 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:55.973 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851474+0000", 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2139999999999999e-06 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851477+0000", 2024-09-14T07:00:55.974 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.208e-06 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851480+0000", 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4700000000000001e-06 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852248+0000", 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00076849899999999996 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852257+0000", 2024-09-14T07:00:55.975 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6840000000000002e-06 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597699+0000", 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74544160400000004 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597706+0000", 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4170000000000003e-06 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.976 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251171+0000", 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65346452 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251284+0000", 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113498 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172760 3.8 3:16edf585:::rbd_data.114fd4a72a77.00000000000001b6:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:55.977 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.030796789, 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3999111539999998, 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172760 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:55.978 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.979 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851497+0000", 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1209999999999999e-06 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851501+0000", 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3279999999999999e-06 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851505+0000", 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.275e-06 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.980 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852297+0000", 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079227500000000005 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852306+0000", 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6659999999999997e-06 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597736+0000", 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74543018500000002 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.981 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597743+0000", 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4290000000000004e-06 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251309+0000", 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6535660569999999 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251407+0000", 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7603999999999996e-05 2024-09-14T07:00:55.982 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172775 3.8 3:104e7293:::rbd_data.114fd4a72a77.00000000000001c5:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.030768718, 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4000188580000001, 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172775 2024-09-14T07:00:55.983 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.984 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6100000000000002e-07 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851531+0000", 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2760000000000003e-06 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851536+0000", 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2640000000000002e-06 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.985 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851538+0000", 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5289999999999998e-06 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852358+0000", 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081989599999999997 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852368+0000", 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597767+0000", 2024-09-14T07:00:55.986 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74539930200000004 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597771+0000", 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4039999999999997e-06 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251441+0000", 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6536695479999999 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.987 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251543+0000", 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000101919 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172778 3.8 3:12efabcb:::rbd_data.114fd4a72a77.00000000000001c8:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0307506069999999, 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400145024, 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.988 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172778 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3900000000000001e-07 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851543+0000", 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851546+0000", 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851549+0000", 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4530000000000001e-06 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852399+0000", 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00085052200000000004 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852407+0000", 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.238e-06 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597814+0000", 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74540710700000001 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597821+0000", 2024-09-14T07:00:55.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251576+0000", 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653755171 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251687+0000", 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011061 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172814 3.8 3:166e6639:::rbd_data.114fd4a72a77.00000000000001ec:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 6.0307406019999998, 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400279984, 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172814 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:55.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592075+0000", 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74052290899999995 2024-09-14T07:00:55.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592081+0000", 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6339999999999998e-06 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592090+0000", 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5929999999999995e-06 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592097+0000", 2024-09-14T07:00:55.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4930000000000003e-06 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597854+0000", 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0057574890000000002 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597858+0000", 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.228e-06 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:55.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251725+0000", 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6538667419999999 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251832+0000", 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000106896 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:55.998 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:56.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:56.000 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:55.999+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:56.101 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:56.100+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:56.113 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:56.113 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:56.113 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:56.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:56.202 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:56.201+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:56.269 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:56.269 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:56.269 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:56.270 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:56.270 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:56.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:00:56.322 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:56.322+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:56.387 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.982037316000003, 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:00:56.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:00:56.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:00:56.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:56.393 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981975476000002, 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:56.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.9819423, 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:00:56.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:00:56.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:00:56.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:00:56.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:00:56.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:00:56.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:00:56.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:00:56.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981819428999998, 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981784679999997, 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:00:56.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:00:56.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981760752, 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:56.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:00:56.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:00:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981738518, 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:00:56.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:00:56.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981705550000001, 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981354779, 2024-09-14T07:00:56.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:00:56.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:00:56.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981250596000002, 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981210085999997, 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:00:56.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:00:56.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:00:56.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:56.442 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981184966000001, 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:00:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:00:56.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.447 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.981030668999999, 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:00:56.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.980997672999997, 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:00:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.980974768000003, 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:00:56.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:00:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:00:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:00:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:00:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.980924764999997, 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:00:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:00:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:00:56.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:00:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:00:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:00:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:00:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.786192712999998, 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:00:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:00:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.781486199, 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:00:56.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:56.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:56.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.778102533999999, 2024-09-14T07:00:56.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:00:56.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:00:56.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.778082799000003, 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:56.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:00:56.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:00:56.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:00:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:56.487 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:56.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:00:56.490 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:56.488+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:56.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:00:56.621 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:56.619+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:56.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:00:56.647 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:56.647 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:56.647 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:00:56.647 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:56.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:00:56.691 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:56.690+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:56.721 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:56.739 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:56.739 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:56.757 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:56.757 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:56.774 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:56.774 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:56.791 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:56.792 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:56.791+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:56.809 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:56.809 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:56.816 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:56.816 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:56.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:00:56.826 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:56.844 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:56.897 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:56.896+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:56.934 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:56.935 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:56.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:00:57.028 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:57.027+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:57.091 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:57.091 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:57.094 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:57.093+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:57.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:00:57.195 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:57.195+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:57.247 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:57.247 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:57.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:00:57.301 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:57.300+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:57.363 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:57.363 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:57.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:00:57.413 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:57.412+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:57.498 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:57.498+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:57.519 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:57.519 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:57.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:00:57.600 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:57.599+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:57.624 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T07:00:57.684 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:57.684 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:57.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:00:57.700 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:57.699+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:57.800 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:57.799+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:57.807 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:57.807 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:57.816 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:00:57.900 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:57.900+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:57.921 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:57.921 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:57.929 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:00:58.013 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:58.013+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:58.074 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:58.074 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:58.082 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:00:58.124 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:58.123+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:58.203 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:58.203+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:58.228 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:58.228 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:58.237 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:00:58.305 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:58.304+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:58.380 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:58.381 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:58.389 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:00:58.405 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:58.405+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:58.492 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:58.492 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:58.501 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:00:58.506 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:58.505+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:58.605 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:58.605 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:58.607 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:58.606+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:58.613 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:00:58.708 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:58.708+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:58.735 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:58.735 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:58.745 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:00:58.829 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:58.829+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:58.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:00:58.890 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:58.890 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:58.899 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:00:58.940 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:00:58.939+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:00:58.983 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:00:58.983 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:00:59.000 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:00:59.000 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:00:59.011 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:00:59.010+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:00:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:00:59.017 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:00:59.033 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:00:59.033 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:00:59.043 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:00:59.043 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:00:59.050 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:00:59.050 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:00:59.052 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:00:59.068 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:00:59.068 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:00:59.085 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:00:59.085 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:00:59.102 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:00:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:00:59.111 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:59.111+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:59.194 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:59.194 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:00:59.194 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:59.203 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:00:59.247 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:00:59.247+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:00:59.305 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:59.306 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:00:59.306 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:00:59.306 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:00:59.306 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:59.313 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:59.312+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:00:59.314 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:00:59.414 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:59.413+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:59.459 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172493 3.8 3:11834377:::rbd_data.114fd4a72a77.0000000000000129:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6066885519999996, 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.395516846, 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.460 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172493 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851241+0000", 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164e-06 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851244+0000", 2024-09-14T07:00:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1259999999999998e-06 2024-09-14T07:00:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851247+0000", 2024-09-14T07:00:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T07:00:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851660+0000", 2024-09-14T07:00:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041335800000000001 2024-09-14T07:00:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851667+0000", 2024-09-14T07:00:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9809999999999997e-06 2024-09-14T07:00:59.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597160+0000", 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549261700000002 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597167+0000", 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2629999999999997e-06 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246685+0000", 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649518327 2024-09-14T07:00:59.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246756+0000", 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1210000000000004e-05 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172499 3.8 3:1336cd84:::rbd_data.114fd4a72a77.000000000000012f:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6066742450000007, 2024-09-14T07:00:59.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3956566389999998, 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172499 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.22e-07 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851255+0000", 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.114e-06 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851259+0000", 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2909999999999999e-06 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851261+0000", 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.655e-06 2024-09-14T07:00:59.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851691+0000", 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043023900000000001 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851695+0000", 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4000000000000001e-06 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597190+0000", 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745495146 2024-09-14T07:00:59.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597194+0000", 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7050000000000001e-06 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246801+0000", 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649607445 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246911+0000", 2024-09-14T07:00:59.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000109422 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172507 3.8 3:17c8295f:::rbd_data.114fd4a72a77.0000000000000136:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6066636929999998, 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3957446349999998, 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.472 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172507 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.473 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:00:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851265+0000", 2024-09-14T07:00:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:59.474 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851266+0000", 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851269+0000", 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0079999999999998e-06 2024-09-14T07:00:59.475 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851271+0000", 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.531e-06 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851725+0000", 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00045350300000000001 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.476 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851732+0000", 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9600000000000003e-06 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597223+0000", 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745490926 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597230+0000", 2024-09-14T07:00:59.477 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4499999999999998e-06 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246943+0000", 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649713363 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247009+0000", 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5656999999999998e-05 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.478 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172606 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 14336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6066534850000007, 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.39623221, 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.479 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172606 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.480 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851276+0000", 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9399999999999993e-07 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.481 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851279+0000", 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0070000000000002e-06 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851282+0000", 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9210000000000001e-06 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.482 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851756+0000", 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000474005 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851759+0000", 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3670000000000001e-06 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597256+0000", 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549659599999996 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597260+0000", 2024-09-14T07:00:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:00:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247047+0000", 2024-09-14T07:00:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649786975 2024-09-14T07:00:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247140+0000", 2024-09-14T07:00:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3449999999999995e-05 2024-09-14T07:00:59.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247507+0000", 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000366664 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172607 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 40960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6066361990000004, 2024-09-14T07:00:59.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3965954819999999, 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172607 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:59.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7000000000000001e-07 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851294+0000", 2024-09-14T07:00:59.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3710000000000001e-06 2024-09-14T07:00:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851297+0000", 2024-09-14T07:00:59.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7850000000000002e-06 2024-09-14T07:00:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851309+0000", 2024-09-14T07:00:59.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1148000000000001e-05 2024-09-14T07:00:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851788+0000", 2024-09-14T07:00:59.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00047951899999999997 2024-09-14T07:00:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851803+0000", 2024-09-14T07:00:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5074e-05 2024-09-14T07:00:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597290+0000", 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548693700000002 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597298+0000", 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3590000000000008e-06 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247532+0000", 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.650233759 2024-09-14T07:00:59.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247590+0000", 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8091000000000003e-05 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247887+0000", 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029716900000000001 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172608 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 67584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:00:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6066078539999999, 2024-09-14T07:00:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3970747800000001, 2024-09-14T07:00:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.495 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172608 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851321+0000", 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0600000000000001e-07 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851322+0000", 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.751e-06 2024-09-14T07:00:59.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851331+0000", 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1799999999999996e-06 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851334+0000", 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3409999999999999e-06 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851830+0000", 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00049571000000000001 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851833+0000", 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6069999999999999e-06 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597326+0000", 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549257700000005 2024-09-14T07:00:59.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597333+0000", 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247917+0000", 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6505841430000001 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248020+0000", 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102581 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248395+0000", 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037526200000000001 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172609 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 94208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.501 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6065882079999998, 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3975014290000001, 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172609 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9200000000000002e-07 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851342+0000", 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.451e-06 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851346+0000", 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9999999999999998e-06 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851348+0000", 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5780000000000001e-06 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851868+0000", 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051942399999999999 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851876+0000", 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6850000000000007e-06 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597371+0000", 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745494502 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597378+0000", 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6129999999999998e-06 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248428+0000", 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.651049502 2024-09-14T07:00:59.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248482+0000", 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3767999999999997e-05 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248841+0000", 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00035961400000000002 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.507 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172610 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 120832~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6065674039999998, 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3980250019999998, 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.508 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172610 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:59.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6800000000000002e-07 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851362+0000", 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4300000000000001e-06 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851366+0000", 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.811e-06 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851369+0000", 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4739999999999999e-06 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851901+0000", 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053255399999999997 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851905+0000", 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4170000000000001e-06 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597404+0000", 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549902099999998 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597408+0000", 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.527e-06 2024-09-14T07:00:59.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248892+0000", 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65148359 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249005+0000", 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113397 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249386+0000", 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038051300000000002 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172611 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 147456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6065491260000009, 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3985748930000002, 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.514 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172611 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:59.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.79e-07 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851381+0000", 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7379999999999999e-06 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851385+0000", 2024-09-14T07:00:59.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5750000000000001e-06 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851387+0000", 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851934+0000", 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00054657299999999996 2024-09-14T07:00:59.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851942+0000", 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2549999999999997e-06 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597437+0000", 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549549800000003 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597445+0000", 2024-09-14T07:00:59.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5839999999999997e-06 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249454+0000", 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6520086940000001 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249576+0000", 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012215899999999999 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249954+0000", 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037802300000000001 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172612 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 174080~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6065375290000006, 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3988136949999999, 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.520 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172612 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:59.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851392+0000", 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851395+0000", 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2449999999999998e-06 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851398+0000", 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4660000000000002e-06 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851978+0000", 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058031599999999999 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851984+0000", 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1319999999999999e-06 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597469+0000", 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548469799999995 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597473+0000", 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8510000000000001e-06 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249990+0000", 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6525175509999999 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250091+0000", 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010111900000000001 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250204+0000", 2024-09-14T07:00:59.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113034 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172613 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 200704~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6065275559999996, 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3989707839999999, 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.526 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172613 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.05e-07 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851402+0000", 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.026e-06 2024-09-14T07:00:59.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851405+0000", 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2440000000000002e-06 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851407+0000", 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3920000000000001e-06 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852016+0000", 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060825699999999996 2024-09-14T07:00:59.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852024+0000", 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0639999999999994e-06 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597505+0000", 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548078699999998 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597513+0000", 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1659999999999994e-06 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250226+0000", 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652713648 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250279+0000", 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2880999999999997e-05 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250371+0000", 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2114000000000006e-05 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172614 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 227328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6065177459999997, 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3991723779999998, 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172614 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.532 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851411+0000", 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T07:00:59.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851412+0000", 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4900000000000004e-07 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851415+0000", 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.942e-06 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851417+0000", 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000002e-06 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852054+0000", 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00063717699999999995 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852058+0000", 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6739999999999999e-06 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597543+0000", 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548541400000001 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597548+0000", 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6140000000000002e-06 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250407+0000", 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652859388 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250487+0000", 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9523000000000004e-05 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250583+0000", 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6026999999999994e-05 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172615 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 253952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6065080869999999, 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3993917979999999, 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172615 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.538 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:00:59.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851421+0000", 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669999999999999e-06 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851425+0000", 2024-09-14T07:00:59.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0110000000000002e-06 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851428+0000", 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.452e-06 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852092+0000", 2024-09-14T07:00:59.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000664497 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852099+0000", 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1199999999999996e-06 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597585+0000", 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548563599999995 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597593+0000", 2024-09-14T07:00:59.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7989999999999994e-06 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250631+0000", 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653038477 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250689+0000", 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7192999999999999e-05 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250812+0000", 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000123344 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172719 3.8 3:10ac98f7:::rbd_data.114fd4a72a77.000000000000018d:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6064972710000003, 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3995141609999999, 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.544 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172719 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851432+0000", 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:00:59.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851435+0000", 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1980000000000001e-06 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851442+0000", 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9419999999999998e-06 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852133+0000", 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069105100000000004 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852139+0000", 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597622+0000", 2024-09-14T07:00:59.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548300599999995 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597635+0000", 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3183000000000001e-05 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250847+0000", 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6532118730000001 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250945+0000", 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7770999999999996e-05 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172725 3.8 3:16c5e555:::rbd_data.114fd4a72a77.0000000000000193:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6064706320000006, 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3996608569999998, 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172725 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851459+0000", 2024-09-14T07:00:59.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.626e-06 2024-09-14T07:00:59.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851466+0000", 2024-09-14T07:00:59.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8349999999999998e-06 2024-09-14T07:00:59.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851469+0000", 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852192+0000", 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072263799999999995 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852201+0000", 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2129999999999997e-06 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597665+0000", 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74546438299999995 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597673+0000", 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6410000000000002e-06 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251000+0000", 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653327011 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251118+0000", 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000118446 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172726 3.8 3:143e5002:::rbd_data.114fd4a72a77.0000000000000194:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6064553769999996, 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3998113490000001, 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172726 2024-09-14T07:00:59.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:00:59.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851474+0000", 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2139999999999999e-06 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851477+0000", 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.208e-06 2024-09-14T07:00:59.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851480+0000", 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4700000000000001e-06 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852248+0000", 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00076849899999999996 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852257+0000", 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6840000000000002e-06 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597699+0000", 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74544160400000004 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597706+0000", 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4170000000000003e-06 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251171+0000", 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65346452 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251284+0000", 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113498 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172760 3.8 3:16edf585:::rbd_data.114fd4a72a77.00000000000001b6:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6064323510000005, 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3999111539999998, 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172760 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:00:59.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851497+0000", 2024-09-14T07:00:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1209999999999999e-06 2024-09-14T07:00:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851501+0000", 2024-09-14T07:00:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3279999999999999e-06 2024-09-14T07:00:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851505+0000", 2024-09-14T07:00:59.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.275e-06 2024-09-14T07:00:59.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852297+0000", 2024-09-14T07:00:59.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079227500000000005 2024-09-14T07:00:59.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852306+0000", 2024-09-14T07:00:59.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6659999999999997e-06 2024-09-14T07:00:59.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597736+0000", 2024-09-14T07:00:59.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74543018500000002 2024-09-14T07:00:59.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597743+0000", 2024-09-14T07:00:59.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4290000000000004e-06 2024-09-14T07:00:59.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251309+0000", 2024-09-14T07:00:59.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6535660569999999 2024-09-14T07:00:59.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251407+0000", 2024-09-14T07:00:59.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7603999999999996e-05 2024-09-14T07:00:59.577 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.577 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.578 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.578 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172775 3.8 3:104e7293:::rbd_data.114fd4a72a77.00000000000001c5:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.578 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6064042799999996, 2024-09-14T07:00:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4000188580000001, 2024-09-14T07:00:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.579 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.580 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.580 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.580 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172775 2024-09-14T07:00:59.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.580 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6100000000000002e-07 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851531+0000", 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2760000000000003e-06 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851536+0000", 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2640000000000002e-06 2024-09-14T07:00:59.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851538+0000", 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5289999999999998e-06 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852358+0000", 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081989599999999997 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852368+0000", 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:00:59.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597767+0000", 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74539930200000004 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597771+0000", 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4039999999999997e-06 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251441+0000", 2024-09-14T07:00:59.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6536695479999999 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251543+0000", 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000101919 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172778 3.8 3:12efabcb:::rbd_data.114fd4a72a77.00000000000001c8:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6063861690000003, 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400145024, 2024-09-14T07:00:59.585 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172778 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3900000000000001e-07 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851543+0000", 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851546+0000", 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851549+0000", 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4530000000000001e-06 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852399+0000", 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00085052200000000004 2024-09-14T07:00:59.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852407+0000", 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.238e-06 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597814+0000", 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74540710700000001 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597821+0000", 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251576+0000", 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653755171 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251687+0000", 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011061 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172814 3.8 3:166e6639:::rbd_data.114fd4a72a77.00000000000001ec:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 9.6063761640000003, 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400279984, 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:00:59.591 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172814 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592075+0000", 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74052290899999995 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592081+0000", 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6339999999999998e-06 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592090+0000", 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5929999999999995e-06 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592097+0000", 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4930000000000003e-06 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597854+0000", 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0057574890000000002 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597858+0000", 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.228e-06 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251725+0000", 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6538667419999999 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251832+0000", 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000106896 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:00:59.596 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:00:59.596 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:00:59.598 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:00:59.597+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:00:59.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:00:59.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:00:59.698 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:00:59.698+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:00:59.727 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4431.0:191975 3.3 e103/99 3:c27dd3c2:::rbd_data.114f2b0c87e5.0000000000000023:head v 103'616, mlcod=103'613)", 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:59.725396+0000", 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0018640690000000001, 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018860680000000001, 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.725396+0000", 2024-09-14T07:00:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.725396+0000", 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.725396+0000", 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0999999999999998e-07 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.725402+0000", 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3530000000000002e-06 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.725410+0000", 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9959999999999993e-06 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.725417+0000", 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7819999999999996e-06 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.725509+0000", 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2419999999999999e-05 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.725528+0000", 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9238000000000001e-05 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4431.0:191980 3.13 e103/24 3:cf5e9b4e:::rbd_data.114f2b0c87e5.0000000000000028:head v 103'507, mlcod=103'505)", 2024-09-14T07:00:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:59.726689+0000", 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.00057099600000000005, 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00065661300000000001, 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.726689+0000", 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.726689+0000", 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.726690+0000", 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9199999999999999e-07 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.726694+0000", 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2390000000000004e-06 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.726705+0000", 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0841e-05 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.726711+0000", 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.726783+0000", 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1414000000000004e-05 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.726802+0000", 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8774999999999999e-05 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4431.0:191979 3.1b e103/24)", 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:59.727280+0000", 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.99998, 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000115395, 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:00:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.727280+0000", 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.727280+0000", 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.727280+0000", 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9400000000000005e-07 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.727286+0000", 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0470000000000001e-06 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.727298+0000", 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2225000000000001e-05 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:00:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:00:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:59.727305+0000", 2024-09-14T07:00:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2130000000000002e-06 2024-09-14T07:00:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:00:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:00:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:00:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 3 2024-09-14T07:00:59.738 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:59.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:00:59.828 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:00:59.827+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:00:59.894 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:00:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:00:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:00:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:00:59.895 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:00:59.900 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:00:59.899+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:00:59.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:01:00.000 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:00:59.999+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.647152052000003, 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:01:00.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:01:00.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:01:00.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.647090212000002, 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:01:00.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.062 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.647057036, 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:00.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646934164999998, 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:01:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:01:00.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646899415999997, 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:00.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646875487999999, 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:01:00.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:01:00.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646853254, 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:01:00.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646820286000001, 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:01:00.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646469515, 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.092 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:01:00.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:01:00.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646365332000002, 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:01:00.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:00.101 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646324821999997, 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:00.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646299702, 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:00.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:01:00.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646145404999999, 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:01:00.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:01:00.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646112408999997, 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:01:00.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.120 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646089504000003, 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.646039500999997, 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:01:00.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.451307448999998, 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:00.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.446600934999999, 2024-09-14T07:01:00.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:01:00.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.443217269999998, 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:01:00.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.443197535000003, 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:01:00.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.148 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:00.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:01:00.150 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:00.148+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:00.249 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:00.248+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:00.297 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:00.298 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:00.298 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:00.298 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:00.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:01:00.350 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:00.350+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:00.413 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:00.413 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:00.414 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:00.414 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:00.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:01:00.463 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:00.462+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:00.553 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:00.552+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 106.835879582, 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:01:00.571 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:01:00.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:01:00.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:00.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:176998 3.1c 3:3b6a06c0:::rbd_data.114f2b0c87e5.00000000000000af:21 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.678122343, 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030101810999999999, 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176998 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1099999999999995e-07 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891479+0000", 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891486+0000", 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7449999999999995e-06 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891493+0000", 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4829999999999996e-06 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905071+0000", 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013577878999999999 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905181+0000", 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110005 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.921576+0000", 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016394855 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177029 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.677580698, 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027087837, 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177029 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892022+0000", 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7780000000000002e-06 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892026+0000", 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892035+0000", 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7169999999999997e-06 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892041+0000", 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.918935+0000", 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026893675999999998 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919012+0000", 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7599e-05 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919103+0000", 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0734000000000002e-05 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177030 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 40960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.677537392, 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027219149000000002, 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177030 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.587 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3300000000000002e-07 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892062+0000", 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0259999999999999e-06 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892069+0000", 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1409999999999999e-06 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892075+0000", 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5330000000000002e-06 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919128+0000", 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027053549999999999 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919196+0000", 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7316000000000004e-05 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919278+0000", 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2050000000000002e-05 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177031 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.677506019999999, 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027358328000000001, 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177031 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892091+0000", 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9900000000000001e-07 2024-09-14T07:01:00.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892093+0000", 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7180000000000001e-06 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892100+0000", 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7630000000000003e-06 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892106+0000", 2024-09-14T07:01:00.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2589999999999997e-06 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919302+0000", 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027195568 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919368+0000", 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6030000000000003e-05 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919448+0000", 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0490999999999999e-05 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177032 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.677474737000001, 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027491581000000001, 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177032 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892122+0000", 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892124+0000", 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3010000000000001e-06 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892131+0000", 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8600000000000004e-06 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892137+0000", 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5690000000000004e-06 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919472+0000", 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027335818000000001 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919539+0000", 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6610000000000001e-05 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919613+0000", 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3930000000000005e-05 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177033 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.677450148, 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027624723, 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177033 2024-09-14T07:01:00.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892148+0000", 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0889999999999998e-06 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892157+0000", 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9719999999999998e-06 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892163+0000", 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1240000000000004e-06 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919636+0000", 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027473329000000001 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919698+0000", 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2385999999999995e-05 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919771+0000", 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2384000000000004e-05 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177034 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 147456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.677424870999999, 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027770280000000001, 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177034 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892172+0000", 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892174+0000", 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6620000000000001e-06 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892180+0000", 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0079999999999999e-06 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892185+0000", 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0490000000000003e-06 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919795+0000", 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027610151999999999 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919864+0000", 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.834e-05 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919941+0000", 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7627999999999996e-05 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177035 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 174080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.677394042, 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027885375, 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177035 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892205+0000", 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5239999999999999e-06 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892211+0000", 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749999999999999e-06 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892217+0000", 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3399999999999997e-06 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919964+0000", 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027746815000000001 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920025+0000", 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1424e-05 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920087+0000", 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2438000000000001e-05 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177036 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 200704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.677371751999999, 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028015455000000002, 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177036 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892225+0000", 2024-09-14T07:01:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892227+0000", 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1579999999999999e-06 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892241+0000", 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4353e-05 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892248+0000", 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4230000000000002e-06 2024-09-14T07:01:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920111+0000", 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027863849 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920176+0000", 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4873e-05 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920240+0000", 2024-09-14T07:01:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3377e-05 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177037 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 227328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.67733832, 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028139249000000002, 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177037 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892261+0000", 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4889999999999998e-06 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892268+0000", 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7939999999999997e-06 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892290+0000", 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2444999999999999e-05 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920265+0000", 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027974987 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920332+0000", 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6935999999999997e-05 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920397+0000", 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5081999999999998e-05 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177038 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.677296145, 2024-09-14T07:01:00.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028249414, 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177038 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2900000000000004e-07 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892303+0000", 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6790000000000001e-06 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892312+0000", 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2839999999999995e-06 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892318+0000", 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1460000000000004e-06 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920421+0000", 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028103197999999999 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920486+0000", 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4960000000000001e-05 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920549+0000", 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3618e-05 2024-09-14T07:01:00.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177325 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.668792279, 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026889606999999999, 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177325 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900808+0000", 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-14T07:01:00.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900816+0000", 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4409999999999993e-06 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900820+0000", 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927681+0000", 2024-09-14T07:01:00.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026860346 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927693+0000", 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2685e-05 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177326 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:00.630 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.668770742, 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026903018000000001, 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177326 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900826+0000", 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900827+0000", 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.544e-06 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900845+0000", 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7484999999999999e-05 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900849+0000", 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5630000000000002e-06 2024-09-14T07:01:00.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927720+0000", 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02687121 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927728+0000", 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8800000000000008e-06 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177327 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.668740392, 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026904081999999999, 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177327 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900858+0000", 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.962e-06 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900869+0000", 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.148e-05 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900874+0000", 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2250000000000002e-06 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927752+0000", 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026878655000000001 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927760+0000", 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3719999999999999e-06 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177365 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:00.638 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.668385468, 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026950818000000001, 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177365 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0100000000000001e-07 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901212+0000", 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3799999999999999e-06 2024-09-14T07:01:00.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901221+0000", 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5809999999999997e-06 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901225+0000", 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928151+0000", 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026925722999999999 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928161+0000", 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0596e-05 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177366 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.66836631, 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026960812000000001, 2024-09-14T07:01:00.642 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177366 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.91e-07 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901231+0000", 2024-09-14T07:01:00.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.409e-06 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901240+0000", 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5290000000000008e-06 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901244+0000", 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1250000000000003e-06 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928182+0000", 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026937492 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928191+0000", 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.646 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177367 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.668347049999999, 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026971434999999998, 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177367 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:01:00.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901251+0000", 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6750000000000001e-06 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901259+0000", 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901263+0000", 2024-09-14T07:01:00.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1119999999999998e-06 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928213+0000", 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026949533000000001 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928220+0000", 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7800000000000001e-06 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177404 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.667962914, 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026884965, 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177404 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901635+0000", 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5710000000000001e-06 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901644+0000", 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6969999999999999e-06 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901648+0000", 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0690000000000003e-06 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928511+0000", 2024-09-14T07:01:00.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026863267999999999 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928518+0000", 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0600000000000002e-06 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177405 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.667943424000001, 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026894992, 2024-09-14T07:01:00.654 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177405 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:00.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901654+0000", 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901663+0000", 2024-09-14T07:01:00.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0280000000000006e-06 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901668+0000", 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1160000000000001e-06 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928541+0000", 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026873107 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928548+0000", 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.973e-06 2024-09-14T07:01:00.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:00.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:00.658 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:00.658 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:01:00.658 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:01:00.658 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-14T07:01:00.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-14T07:01:00.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:01:00.660 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T07:01:00.661 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:00.658+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:00.761 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:00.759+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:00.814 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-14T07:01:00.814 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-14T07:01:00.815 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-14T07:01:00.818 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:01:00.818 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:01:00.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-14T07:01:00.821 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:00.821 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:00.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:01:00.871 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:00.870+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:00.956 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-14T07:01:00.956 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-14T07:01:00.956 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-14T07:01:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-14T07:01:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-14T07:01:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-14T07:01:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-14T07:01:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:01:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-14T07:01:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-14T07:01:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-14T07:01:00.957 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-14T07:01:00.959 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:01:00.959 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:01:00.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-14T07:01:00.962 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:00.961+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:00.979 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:00.979 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:00.980 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:00.980 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:00.980 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:00.989 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:01:01.072 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:01.072+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-14T07:01:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-14T07:01:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-14T07:01:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-14T07:01:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-14T07:01:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-14T07:01:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-14T07:01:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:01:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-14T07:01:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-14T07:01:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-14T07:01:01.099 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-14T07:01:01.101 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:01:01.101 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:01:01.102 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-14T07:01:01.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:01:01.139 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:01.139 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:01.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:01.139 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119746 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1298432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.592891205999997, 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067798875999999994, 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.140 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119746 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545019+0000", 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545020+0000", 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.782e-06 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545026+0000", 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9390000000000002e-06 2024-09-14T07:01:01.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545031+0000", 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5770000000000004e-06 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612601+0000", 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067570682000000007 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612683+0000", 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1378999999999995e-05 2024-09-14T07:01:01.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612817+0000", 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013415000000000001 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119747 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.592872786999997, 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068036012000000007, 2024-09-14T07:01:01.144 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119747 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546792+0000", 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017554960000000001 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546797+0000", 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1950000000000002e-06 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546808+0000", 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0675e-05 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546815+0000", 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612854+0000", 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066039158000000001 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612930+0000", 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.613073+0000", 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000142798 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119754 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1413120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.148 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.590858087000001, 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067735654000000006, 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119754 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549973+0000", 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029215999999999999 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549978+0000", 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3510000000000001e-06 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549988+0000", 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0037000000000001e-05 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549995+0000", 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9709999999999998e-06 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614569+0000", 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064573368000000006 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614650+0000", 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1909000000000005e-05 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614787+0000", 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013641799999999999 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.152 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119764 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1556480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.587686179999999, 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067672107999999995, 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.153 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119764 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550224+0000", 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550226+0000", 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1720000000000001e-06 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550233+0000", 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9879999999999998e-06 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550238+0000", 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1819999999999997e-06 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617715+0000", 2024-09-14T07:01:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067476628999999996 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617807+0000", 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2899999999999995e-05 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617895+0000", 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7837999999999995e-05 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119765 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.587663747999997, 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067829500000000001, 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119765 2024-09-14T07:01:01.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7e-07 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550248+0000", 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1280000000000002e-06 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550256+0000", 2024-09-14T07:01:01.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550261+0000", 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617926+0000", 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067665280999999994 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617995+0000", 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9478000000000001e-05 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618075+0000", 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9666000000000006e-05 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119950 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 529408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.162 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.587641642000001, 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067992002999999995, 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119950 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550852+0000", 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058435400000000004 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550857+0000", 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6979999999999997e-06 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550867+0000", 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0378e-05 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550874+0000", 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8480000000000003e-06 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618106+0000", 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067231714999999997 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618178+0000", 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2572999999999998e-05 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618260+0000", 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1437e-05 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119970 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 816128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580986486, 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067745828999999994, 2024-09-14T07:01:01.167 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119970 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556924+0000", 2024-09-14T07:01:01.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0410000000000001e-06 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556927+0000", 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9679999999999998e-06 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556930+0000", 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624487+0000", 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067557392999999993 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624551+0000", 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3861000000000004e-05 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624669+0000", 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117921 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119971 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580976546999999, 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067869065000000006, 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119971 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556934+0000", 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.096e-06 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556937+0000", 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.796e-06 2024-09-14T07:01:01.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556939+0000", 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624708+0000", 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067768992 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624747+0000", 2024-09-14T07:01:01.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8148999999999997e-05 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624802+0000", 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.533e-05 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119972 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 844800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580966875999998, 2024-09-14T07:01:01.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068010762000000002, 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119972 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:01.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556943+0000", 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556944+0000", 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556947+0000", 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9720000000000001e-06 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556949+0000", 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4899999999999999e-06 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624838+0000", 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067888899000000003 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624895+0000", 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7395000000000002e-05 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624953+0000", 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7745999999999999e-05 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119973 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 859136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580956993999997, 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068210898000000006, 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119973 2024-09-14T07:01:01.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556953+0000", 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556954+0000", 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189999999999999e-06 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556957+0000", 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9490000000000001e-06 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556959+0000", 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3870000000000002e-06 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624983+0000", 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068023726000000007 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625049+0000", 2024-09-14T07:01:01.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6385999999999998e-05 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625163+0000", 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114221 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119974 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:01.185 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580947072999997, 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068338594000000003, 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119974 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:01.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556963+0000", 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556964+0000", 2024-09-14T07:01:01.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556969+0000", 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.04e-06 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556974+0000", 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.639e-06 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625194+0000", 2024-09-14T07:01:01.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068220742000000001 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625246+0000", 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1844e-05 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625301+0000", 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4551e-05 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.189 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119975 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580928833000002, 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068554198999999996, 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119975 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556983+0000", 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2560000000000001e-06 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556989+0000", 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2210000000000002e-06 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556994+0000", 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7779999999999998e-06 2024-09-14T07:01:01.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625330+0000", 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068335911999999999 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625396+0000", 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6051999999999997e-05 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625535+0000", 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013860200000000001 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119976 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580907838000002, 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068656164000000006, 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119976 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557004+0000", 2024-09-14T07:01:01.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557010+0000", 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1770000000000003e-06 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557016+0000", 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2650000000000004e-06 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625566+0000", 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068550323999999996 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625604+0000", 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8398000000000001e-05 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625658+0000", 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3239000000000003e-05 2024-09-14T07:01:01.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119977 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580827419000002, 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068713522999999999, 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119977 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:01.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557086+0000", 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3569999999999998e-06 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557094+0000", 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6829999999999997e-06 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557101+0000", 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625689+0000", 2024-09-14T07:01:01.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068587882000000003 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625743+0000", 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4082000000000002e-05 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625795+0000", 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2657000000000001e-05 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119978 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 930816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580798827000002, 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068826410000000005, 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.204 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119978 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557113+0000", 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557120+0000", 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1079999999999998e-06 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557125+0000", 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625823+0000", 2024-09-14T07:01:01.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068698928000000006 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625884+0000", 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0056999999999998e-05 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625937+0000", 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3383e-05 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119979 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 945152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580777679999997, 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068940788000000003, 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119979 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5199999999999998e-07 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557134+0000", 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.018e-06 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557140+0000", 2024-09-14T07:01:01.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557145+0000", 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7790000000000003e-06 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625966+0000", 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068820972999999994 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626019+0000", 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3828e-05 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626072+0000", 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3078000000000002e-05 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119980 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580757321999997, 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069053630000000005, 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119980 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.214 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:01.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557159+0000", 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557167+0000", 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999996e-06 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557172+0000", 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626101+0000", 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068928552000000004 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626154+0000", 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3338000000000003e-05 2024-09-14T07:01:01.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626206+0000", 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1657999999999999e-05 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119981 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580728671999999, 2024-09-14T07:01:01.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069157629999999998, 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119981 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557183+0000", 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557190+0000", 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2249999999999997e-06 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557195+0000", 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3120000000000001e-06 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626234+0000", 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069038926 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626286+0000", 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2604e-05 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626338+0000", 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1947000000000003e-05 2024-09-14T07:01:01.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119982 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 988160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 47.580707519000001, 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069364004000000007, 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119982 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560989+0000", 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037867360000000002 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560994+0000", 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561004+0000", 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3680000000000008e-06 2024-09-14T07:01:01.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561011+0000", 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1640000000000004e-06 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626366+0000", 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.065355286999999998 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626433+0000", 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6554e-05 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626566+0000", 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132996 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:191378 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:01.227 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.4696203270000003, 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.95875689900000005, 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 191378 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668290+0000", 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3319999999999999e-06 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668295+0000", 2024-09-14T07:01:01.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7570000000000004e-06 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668306+0000", 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0813e-05 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668311+0000", 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0839999999999999e-06 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668364+0000", 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2673999999999997e-05 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668371+0000", 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624394+0000", 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.95602263300000001 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624558+0000", 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000163525 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624767+0000", 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000209555 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.626401+0000", 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016343110000000001 2024-09-14T07:01:01.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.626918+0000", 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051661100000000005 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.627005+0000", 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6854999999999994e-05 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.627046+0000", 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0970999999999998e-05 2024-09-14T07:01:01.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:01.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:01.234 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:01.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:01:01.237 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:01.236+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:01.336 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:01.335+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:01.364 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:01.364 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:01.376 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-14T07:01:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-14T07:01:01.380 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:01:01.381 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:01:01.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-14T07:01:01.383 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:01.384 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:01.389 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:01.389 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:01.390 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:01.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:01:01.400 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:01.400 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:01.418 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:01.437 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:01.437 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:01.437 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:01.436+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:01.470 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:01.470 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:01.502 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:01.503 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-14T07:01:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-14T07:01:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-14T07:01:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-14T07:01:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-14T07:01:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-14T07:01:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-14T07:01:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:01:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-14T07:01:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-14T07:01:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-14T07:01:01.524 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-14T07:01:01.526 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:01:01.526 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:01:01.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-14T07:01:01.536 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:01.537 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:01.538 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:01.536+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:01.547 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:01.547 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:01.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:01:01.640 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:01.640+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:01.706 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-14T07:01:01.706 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-14T07:01:01.706 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-14T07:01:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-14T07:01:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-14T07:01:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-14T07:01:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-14T07:01:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:01:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-14T07:01:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-14T07:01:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-14T07:01:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-14T07:01:01.710 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:01:01.711 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:01:01.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-14T07:01:01.732 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:01.732 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:01.742 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:01:01.743 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:01.741+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:01.840 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:01.839+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:01.851 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:01.851 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:01.860 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-14T07:01:01.871 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-14T07:01:01.872 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-14T07:01:01.874 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T07:01:01.874 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T07:01:01.943 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:01.942+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:02.003 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:02.003 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:02.011 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:01:02.053 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:02.051+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:02.143 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:02.142+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:02.154 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:02.154 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:02.163 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:01:02.244 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:02.243+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:02.267 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:02.267 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:02.276 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:01:02.344 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:02.343+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:02.379 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:02.379 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:02.387 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:01:02.445 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:02.444+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:02.534 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:02.535 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:02.543 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:01:02.546 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:02.544+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:02.646 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:02.645+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:02.648 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:02.648 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:02.659 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:01:02.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:01:02.747 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:02.746+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:02.808 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:02.808 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:02.817 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:01:02.848 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:02.847+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:02.949 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:02.949+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:02.962 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:02.962 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:02.971 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:01:03.050 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:03.049+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:03.073 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:03.073 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:03.082 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:01:03.151 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:03.149+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:03.184 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:03.184 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:03.193 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:01:03.252 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:03.252+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:03.337 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:03.337 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:03.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:01:03.354 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:03.352+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:03.454 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:03.453+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:03.458 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:03.459 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:03.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:01:03.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:01:03.555 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:03.554+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:03.617 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:03.617 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:03.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:01:03.656 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:03.655+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:03.659 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T07:01:03.690 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:03.712 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:03.713 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:03.746 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:03.757 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:03.755+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:03.776 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.370674335000004, 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:01:03.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.370612495000003, 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:01:03.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:01:03.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.370579319000001, 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:01:03.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:01:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:01:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:01:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:01:03.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.370456447999999, 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:01:03.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:01:03.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.370421699000005, 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:01:03.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:01:03.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:01:03.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.370397771, 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:01:03.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:01:03.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:01:03.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.370375537000001, 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:03.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.370342569000002, 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:01:03.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:01:03.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.369991798000001, 2024-09-14T07:01:03.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:01:03.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.369887614999996, 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:01:03.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.369847105000005, 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:03.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:01:03.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:03.829 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.369821985000002, 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:01:03.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:01:03.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:01:03.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:01:03.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:01:03.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.369667688000007, 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:01:03.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.369634692000005, 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:01:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:01:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.369611786999997, 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:01:03.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:01:03.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 66.369561783999998, 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:01:03.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:01:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.174829731999999, 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:03.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:03.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.170123218000001, 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:03.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.166739552999999, 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.166719817999997, 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:01:03.868 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:03.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:03.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:03.874 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:03.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:01:03.875 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:03.876 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:03.873+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:03.974 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:03.974+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:04.017 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:04.018 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:04.024 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:04.024 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:04.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:01:04.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:04.035 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:04.052 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:04.052 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:04.104 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:04.103+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:04.142 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:04.142 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:04.142 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:04.142 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:04.142 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:04.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:01:04.176 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:04.175+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:04.278 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:04.276+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 110.57345757900001, 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:01:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:01:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:176998 3.1c 3:3b6a06c0:::rbd_data.114f2b0c87e5.00000000000000af:21 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.415700340000001, 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030101810999999999, 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176998 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1099999999999995e-07 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891479+0000", 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891486+0000", 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7449999999999995e-06 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891493+0000", 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4829999999999996e-06 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905071+0000", 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013577878999999999 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905181+0000", 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110005 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.921576+0000", 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016394855 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177029 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.415158695000001, 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027087837, 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177029 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892022+0000", 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7780000000000002e-06 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892026+0000", 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892035+0000", 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7169999999999997e-06 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892041+0000", 2024-09-14T07:01:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.918935+0000", 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026893675999999998 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919012+0000", 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7599e-05 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919103+0000", 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0734000000000002e-05 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177030 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 40960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.415115389, 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027219149000000002, 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177030 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3300000000000002e-07 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892062+0000", 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0259999999999999e-06 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892069+0000", 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1409999999999999e-06 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892075+0000", 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5330000000000002e-06 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919128+0000", 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027053549999999999 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919196+0000", 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7316000000000004e-05 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919278+0000", 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2050000000000002e-05 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177031 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.415084017, 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027358328000000001, 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177031 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892091+0000", 2024-09-14T07:01:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9900000000000001e-07 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892093+0000", 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7180000000000001e-06 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892100+0000", 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7630000000000003e-06 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892106+0000", 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2589999999999997e-06 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919302+0000", 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027195568 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919368+0000", 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6030000000000003e-05 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919448+0000", 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0490999999999999e-05 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177032 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.415052734, 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027491581000000001, 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177032 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892122+0000", 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892124+0000", 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3010000000000001e-06 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892131+0000", 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8600000000000004e-06 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892137+0000", 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5690000000000004e-06 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919472+0000", 2024-09-14T07:01:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027335818000000001 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919539+0000", 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6610000000000001e-05 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919613+0000", 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3930000000000005e-05 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177033 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.415028145000001, 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027624723, 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177033 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892148+0000", 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0889999999999998e-06 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892157+0000", 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9719999999999998e-06 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892163+0000", 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1240000000000004e-06 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919636+0000", 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027473329000000001 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919698+0000", 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2385999999999995e-05 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919771+0000", 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2384000000000004e-05 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177034 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 147456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.415002868, 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027770280000000001, 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177034 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892172+0000", 2024-09-14T07:01:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892174+0000", 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6620000000000001e-06 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892180+0000", 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0079999999999999e-06 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892185+0000", 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0490000000000003e-06 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919795+0000", 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027610151999999999 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919864+0000", 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.834e-05 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919941+0000", 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7627999999999996e-05 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177035 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 174080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.414972039, 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027885375, 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177035 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:01:04.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892205+0000", 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5239999999999999e-06 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892211+0000", 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749999999999999e-06 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892217+0000", 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3399999999999997e-06 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919964+0000", 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027746815000000001 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920025+0000", 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1424e-05 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920087+0000", 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2438000000000001e-05 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177036 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 200704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.414949749, 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028015455000000002, 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177036 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892225+0000", 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892227+0000", 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1579999999999999e-06 2024-09-14T07:01:04.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892241+0000", 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4353e-05 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892248+0000", 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4230000000000002e-06 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920111+0000", 2024-09-14T07:01:04.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027863849 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920176+0000", 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4873e-05 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920240+0000", 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3377e-05 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177037 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 227328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.414916316999999, 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028139249000000002, 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177037 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.359 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:04.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892261+0000", 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4889999999999998e-06 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892268+0000", 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7939999999999997e-06 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892290+0000", 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2444999999999999e-05 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920265+0000", 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027974987 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920332+0000", 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6935999999999997e-05 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920397+0000", 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5081999999999998e-05 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177038 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.414874142, 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028249414, 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177038 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2900000000000004e-07 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892303+0000", 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6790000000000001e-06 2024-09-14T07:01:04.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892312+0000", 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2839999999999995e-06 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892318+0000", 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1460000000000004e-06 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920421+0000", 2024-09-14T07:01:04.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028103197999999999 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920486+0000", 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4960000000000001e-05 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920549+0000", 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3618e-05 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177325 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.406370276000001, 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026889606999999999, 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177325 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:04.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900808+0000", 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900816+0000", 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4409999999999993e-06 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900820+0000", 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927681+0000", 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026860346 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927693+0000", 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2685e-05 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177326 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.406348739, 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026903018000000001, 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177326 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900826+0000", 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900827+0000", 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.544e-06 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900845+0000", 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7484999999999999e-05 2024-09-14T07:01:04.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900849+0000", 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5630000000000002e-06 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927720+0000", 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02687121 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927728+0000", 2024-09-14T07:01:04.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8800000000000008e-06 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177327 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.406318389000001, 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026904081999999999, 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177327 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900858+0000", 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.962e-06 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900869+0000", 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.148e-05 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900874+0000", 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2250000000000002e-06 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927752+0000", 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026878655000000001 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927760+0000", 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3719999999999999e-06 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177365 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.405963464999999, 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026950818000000001, 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:04.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177365 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0100000000000001e-07 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901212+0000", 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3799999999999999e-06 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901221+0000", 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5809999999999997e-06 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901225+0000", 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928151+0000", 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026925722999999999 2024-09-14T07:01:04.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928161+0000", 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0596e-05 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177366 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.405944307, 2024-09-14T07:01:04.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026960812000000001, 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177366 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:04.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.91e-07 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901231+0000", 2024-09-14T07:01:04.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.409e-06 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901240+0000", 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5290000000000008e-06 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901244+0000", 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1250000000000003e-06 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928182+0000", 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026937492 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928191+0000", 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177367 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.388 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.405925047, 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026971434999999998, 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177367 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901251+0000", 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6750000000000001e-06 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901259+0000", 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901263+0000", 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1119999999999998e-06 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928213+0000", 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026949533000000001 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928220+0000", 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7800000000000001e-06 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.392 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177404 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.405540910999999, 2024-09-14T07:01:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026884965, 2024-09-14T07:01:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.393 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177404 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901635+0000", 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5710000000000001e-06 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901644+0000", 2024-09-14T07:01:04.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6969999999999999e-06 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901648+0000", 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0690000000000003e-06 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928511+0000", 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026863267999999999 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928518+0000", 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0600000000000002e-06 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177405 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.405521421, 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026894992, 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177405 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901654+0000", 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901663+0000", 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0280000000000006e-06 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901668+0000", 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1160000000000001e-06 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928541+0000", 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026873107 2024-09-14T07:01:04.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928548+0000", 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.973e-06 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.401 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:04.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:01:04.403 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:04.401+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:04.504 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:04.502+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:04.511 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:04.511 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:04.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:01:04.605 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:04.604+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:04.628 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:04.628 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:04.628 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:04.629 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:04.629 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:04.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:01:04.706 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:04.705+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:04.746 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119746 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1298432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.200117564999999, 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067798875999999994, 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.747 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119746 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:04.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545019+0000", 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545020+0000", 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.782e-06 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545026+0000", 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9390000000000002e-06 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545031+0000", 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5770000000000004e-06 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612601+0000", 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067570682000000007 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612683+0000", 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1378999999999995e-05 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612817+0000", 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013415000000000001 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119747 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.200099145999999, 2024-09-14T07:01:04.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068036012000000007, 2024-09-14T07:01:04.753 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119747 2024-09-14T07:01:04.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.754 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:04.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:04.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546792+0000", 2024-09-14T07:01:04.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017554960000000001 2024-09-14T07:01:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546797+0000", 2024-09-14T07:01:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1950000000000002e-06 2024-09-14T07:01:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546808+0000", 2024-09-14T07:01:04.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0675e-05 2024-09-14T07:01:04.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546815+0000", 2024-09-14T07:01:04.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:01:04.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612854+0000", 2024-09-14T07:01:04.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066039158000000001 2024-09-14T07:01:04.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612930+0000", 2024-09-14T07:01:04.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.613073+0000", 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000142798 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.760 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119754 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1413120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.760 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:04.760 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.198084446000003, 2024-09-14T07:01:04.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067735654000000006, 2024-09-14T07:01:04.760 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119754 2024-09-14T07:01:04.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.761 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:04.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:04.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549973+0000", 2024-09-14T07:01:04.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029215999999999999 2024-09-14T07:01:04.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549978+0000", 2024-09-14T07:01:04.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3510000000000001e-06 2024-09-14T07:01:04.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549988+0000", 2024-09-14T07:01:04.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0037000000000001e-05 2024-09-14T07:01:04.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549995+0000", 2024-09-14T07:01:04.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9709999999999998e-06 2024-09-14T07:01:04.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614569+0000", 2024-09-14T07:01:04.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064573368000000006 2024-09-14T07:01:04.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614650+0000", 2024-09-14T07:01:04.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1909000000000005e-05 2024-09-14T07:01:04.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614787+0000", 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013641799999999999 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119764 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1556480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.194912539000001, 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067672107999999995, 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119764 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550224+0000", 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550226+0000", 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1720000000000001e-06 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550233+0000", 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9879999999999998e-06 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550238+0000", 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1819999999999997e-06 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617715+0000", 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067476628999999996 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617807+0000", 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2899999999999995e-05 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617895+0000", 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7837999999999995e-05 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119765 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.194890106999999, 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067829500000000001, 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119765 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7e-07 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550248+0000", 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1280000000000002e-06 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550256+0000", 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550261+0000", 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617926+0000", 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067665280999999994 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617995+0000", 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9478000000000001e-05 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618075+0000", 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9666000000000006e-05 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.777 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119950 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 529408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.194868001000003, 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067992002999999995, 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119950 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550852+0000", 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058435400000000004 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550857+0000", 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6979999999999997e-06 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550867+0000", 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0378e-05 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550874+0000", 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8480000000000003e-06 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618106+0000", 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067231714999999997 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618178+0000", 2024-09-14T07:01:04.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2572999999999998e-05 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618260+0000", 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1437e-05 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119970 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 816128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188212845000002, 2024-09-14T07:01:04.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067745828999999994, 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119970 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556924+0000", 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0410000000000001e-06 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556927+0000", 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9679999999999998e-06 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556930+0000", 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624487+0000", 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067557392999999993 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624551+0000", 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3861000000000004e-05 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624669+0000", 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117921 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119971 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188202906000001, 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067869065000000006, 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119971 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556934+0000", 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.096e-06 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556937+0000", 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.796e-06 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556939+0000", 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624708+0000", 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067768992 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624747+0000", 2024-09-14T07:01:04.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8148999999999997e-05 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624802+0000", 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.533e-05 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.791 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119972 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 844800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188193235, 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068010762000000002, 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119972 2024-09-14T07:01:04.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556943+0000", 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556944+0000", 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556947+0000", 2024-09-14T07:01:04.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9720000000000001e-06 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556949+0000", 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4899999999999999e-06 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624838+0000", 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067888899000000003 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624895+0000", 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7395000000000002e-05 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624953+0000", 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7745999999999999e-05 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119973 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 859136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:04.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188183352999999, 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068210898000000006, 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119973 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556953+0000", 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556954+0000", 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189999999999999e-06 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556957+0000", 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9490000000000001e-06 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556959+0000", 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3870000000000002e-06 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624983+0000", 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068023726000000007 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625049+0000", 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6385999999999998e-05 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625163+0000", 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114221 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119974 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188173431999999, 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068338594000000003, 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119974 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.801 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556963+0000", 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-14T07:01:04.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556964+0000", 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556969+0000", 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.04e-06 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556974+0000", 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.639e-06 2024-09-14T07:01:04.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625194+0000", 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068220742000000001 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625246+0000", 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1844e-05 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625301+0000", 2024-09-14T07:01:04.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4551e-05 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119975 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188155192000004, 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068554198999999996, 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119975 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556983+0000", 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2560000000000001e-06 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556989+0000", 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2210000000000002e-06 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556994+0000", 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7779999999999998e-06 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625330+0000", 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068335911999999999 2024-09-14T07:01:04.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625396+0000", 2024-09-14T07:01:04.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6051999999999997e-05 2024-09-14T07:01:04.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625535+0000", 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013860200000000001 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119976 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:04.810 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188134196999997, 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068656164000000006, 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119976 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:01:04.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557004+0000", 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557010+0000", 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1770000000000003e-06 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557016+0000", 2024-09-14T07:01:04.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2650000000000004e-06 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625566+0000", 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068550323999999996 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625604+0000", 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8398000000000001e-05 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625658+0000", 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3239000000000003e-05 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119977 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188053777999997, 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068713522999999999, 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119977 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:04.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:04.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557086+0000", 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3569999999999998e-06 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557094+0000", 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6829999999999997e-06 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557101+0000", 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625689+0000", 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068587882000000003 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625743+0000", 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4082000000000002e-05 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625795+0000", 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2657000000000001e-05 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119978 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 930816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188025185999997, 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068826410000000005, 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119978 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:04.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557113+0000", 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557120+0000", 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1079999999999998e-06 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557125+0000", 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625823+0000", 2024-09-14T07:01:04.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068698928000000006 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625884+0000", 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0056999999999998e-05 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625937+0000", 2024-09-14T07:01:04.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3383e-05 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119979 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 945152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.188004038999999, 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068940788000000003, 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119979 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5199999999999998e-07 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557134+0000", 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.018e-06 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557140+0000", 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557145+0000", 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7790000000000003e-06 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625966+0000", 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068820972999999994 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626019+0000", 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3828e-05 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626072+0000", 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3078000000000002e-05 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119980 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.187983680999999, 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069053630000000005, 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119980 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:01:04.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557159+0000", 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557167+0000", 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999996e-06 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557172+0000", 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626101+0000", 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068928552000000004 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626154+0000", 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3338000000000003e-05 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626206+0000", 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1657999999999999e-05 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119981 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.187955031000001, 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069157629999999998, 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119981 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557183+0000", 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557190+0000", 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2249999999999997e-06 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557195+0000", 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3120000000000001e-06 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626234+0000", 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069038926 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626286+0000", 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2604e-05 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626338+0000", 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1947000000000003e-05 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119982 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 988160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.187933878000003, 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069364004000000007, 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119982 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560989+0000", 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037867360000000002 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560994+0000", 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561004+0000", 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3680000000000008e-06 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561011+0000", 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1640000000000004e-06 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626366+0000", 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.065355286999999998 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626433+0000", 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6554e-05 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626566+0000", 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132996 2024-09-14T07:01:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:191378 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.0768466859999997, 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.95875689900000005, 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 191378 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668290+0000", 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3319999999999999e-06 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668295+0000", 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7570000000000004e-06 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668306+0000", 2024-09-14T07:01:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0813e-05 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668311+0000", 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0839999999999999e-06 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668364+0000", 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2673999999999997e-05 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668371+0000", 2024-09-14T07:01:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624394+0000", 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.95602263300000001 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624558+0000", 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000163525 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624767+0000", 2024-09-14T07:01:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000209555 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.626401+0000", 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016343110000000001 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.626918+0000", 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051661100000000005 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.627005+0000", 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6854999999999994e-05 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.627046+0000", 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0970999999999998e-05 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:04.849 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:04.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:01:04.851 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:04.850+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:04.951 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:04.951+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:04.998 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:04.999 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:04.999 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:04.999 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:05.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:01:05.090 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:05.090+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:05.153 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:05.152+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:05.154 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:05.155 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:05.155 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:05.155 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:05.155 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:05.163 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:01:05.253 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:05.253+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:05.320 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:05.320 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:05.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:05.321 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:05.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.321 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:01:05.321 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:05.321 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.564172805, 2024-09-14T07:01:05.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:01:05.321 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:05.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:01:05.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:01:05.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:01:05.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.323 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:05.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:05.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:01:05.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:01:05.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:01:05.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:01:05.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:01:05.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:01:05.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:01:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.965760212999996, 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:01:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.960100569999994, 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:01:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:01:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:01:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.654264291999993, 2024-09-14T07:01:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:01:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.654233500000004, 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:01:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.654210129999996, 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:01:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:01:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:01:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.653403620999995, 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:01:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:01:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.653381228000001, 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:01:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.653361087999997, 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:01:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:01:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:01:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.653210932999997, 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:01:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:01:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:01:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.653155378999998, 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.653122338000003, 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:01:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:01:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.650769252999993, 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:01:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:01:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:01:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:01:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.650746687999998, 2024-09-14T07:01:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:01:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:01:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.649889693000006, 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:01:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.648672672000004, 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:01:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:01:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:01:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:01:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:01:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:01:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.648544831999999, 2024-09-14T07:01:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:01:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.648524738999996, 2024-09-14T07:01:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:01:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:01:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.647497845999993, 2024-09-14T07:01:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:01:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:01:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:01:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138497 3.14 3:298b17d3:::rbd_data.114f74bd3bbf.00000000000002d7:head [set-alloc-hint object_size 262144 write_size 262144,write 218355~43789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-14T07:01:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.999990564000001, 2024-09-14T07:01:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97686345699999999, 2024-09-14T07:01:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138497 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317983+0000", 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318003+0000", 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0610000000000001e-05 2024-09-14T07:01:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318022+0000", 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8627e-05 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318030+0000", 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318101+0000", 2024-09-14T07:01:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1162999999999996e-05 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318108+0000", 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291257+0000", 2024-09-14T07:01:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97314914100000005 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291396+0000", 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013883799999999999 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291573+0000", 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017716800000000001 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.293147+0000", 2024-09-14T07:01:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015740400000000001 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294753+0000", 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00160597 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294792+0000", 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8758999999999999e-05 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294844+0000", 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2188000000000002e-05 2024-09-14T07:01:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:05.432 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:05.433 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:01:05.434 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:05.432+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:05.534 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:05.534+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:05.539 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:05.539 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:05.539 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:05.540 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:05.548 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:01:05.635 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:05.634+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:05.651 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:05.651 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:05.651 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:05.651 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:05.651 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:05.659 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:01:05.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:01:05.736 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:05.734+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:05.764 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:05.764 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131358 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264589796000003, 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030805583, 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.765 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.766 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.766 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.766 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.766 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131358 2024-09-14T07:01:05.766 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.766 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.766 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.766 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.766 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:05.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.767 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.767 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:05.767 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.767 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.767 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498000+0000", 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.768 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498004+0000", 2024-09-14T07:01:05.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2569999999999999e-06 2024-09-14T07:01:05.769 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.769 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.769 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.769 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498007+0000", 2024-09-14T07:01:05.769 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.507e-06 2024-09-14T07:01:05.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.770 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.770 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499958+0000", 2024-09-14T07:01:05.770 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001950989 2024-09-14T07:01:05.770 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.770 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499962+0000", 2024-09-14T07:01:05.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-14T07:01:05.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.771 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.771 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.771 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528792+0000", 2024-09-14T07:01:05.771 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288306389999999 2024-09-14T07:01:05.771 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528804+0000", 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2096000000000001e-05 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.772 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131359 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264570386000003, 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308363920000001, 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.773 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131359 2024-09-14T07:01:05.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.774 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.774 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.774 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.774 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:01:05.774 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.774 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498019+0000", 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:01:05.775 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498020+0000", 2024-09-14T07:01:05.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4529999999999999e-06 2024-09-14T07:01:05.776 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.776 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.776 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.776 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498023+0000", 2024-09-14T07:01:05.776 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3450000000000002e-06 2024-09-14T07:01:05.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498026+0000", 2024-09-14T07:01:05.777 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.661e-06 2024-09-14T07:01:05.777 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.777 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.777 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.777 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499995+0000", 2024-09-14T07:01:05.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00196926 2024-09-14T07:01:05.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.778 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.778 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.778 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500002+0000", 2024-09-14T07:01:05.778 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6930000000000001e-06 2024-09-14T07:01:05.778 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.779 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.779 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528842+0000", 2024-09-14T07:01:05.779 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288401300000001 2024-09-14T07:01:05.779 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.779 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.780 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.780 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528855+0000", 2024-09-14T07:01:05.780 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2595999999999999e-05 2024-09-14T07:01:05.780 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.780 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.780 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.780 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.780 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131360 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264559382999998, 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030841943, 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131360 2024-09-14T07:01:05.781 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.782 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498030+0000", 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498031+0000", 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1119999999999999e-06 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498034+0000", 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.783 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498036+0000", 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.599e-06 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500030+0000", 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001993613 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500034+0000", 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1060000000000002e-06 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.784 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528867+0000", 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288327429999999 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528871+0000", 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.673e-06 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.785 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131361 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264538637999998, 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308644520000001, 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131361 2024-09-14T07:01:05.786 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.787 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498052+0000", 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.24e-06 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498055+0000", 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1609999999999999e-06 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.788 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498058+0000", 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.05e-06 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500066+0000", 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020081169999999998 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500075+0000", 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0559999999999994e-06 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.789 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528903+0000", 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288276810000001 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528915+0000", 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1938e-05 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.790 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131362 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264519857000003, 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030860661, 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131362 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.791 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:01:05.792 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498075+0000", 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5639999999999996e-06 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498079+0000", 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.793 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498081+0000", 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8660000000000002e-06 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500101+0000", 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020191219999999999 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500104+0000", 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6890000000000002e-06 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.794 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528925+0000", 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028820971 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528930+0000", 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3390000000000003e-06 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131363 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.795 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264503609000002, 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030874107, 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131363 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.796 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.797 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498086+0000", 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1009999999999999e-06 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498090+0000", 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5609999999999999e-06 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.798 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498093+0000", 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7360000000000001e-06 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500136+0000", 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020436030000000002 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500145+0000", 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9530000000000005e-06 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.799 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528948+0000", 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028803103 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528959+0000", 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0808999999999999e-05 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.800 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131364 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264487228, 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308734129999999, 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131364 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.801 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498102+0000", 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498103+0000", 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498107+0000", 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4089999999999999e-06 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498110+0000", 2024-09-14T07:01:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7760000000000002e-06 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500177+0000", 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020678200000000002 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500183+0000", 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4979999999999997e-06 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.804 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528970+0000", 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287867049999999 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528975+0000", 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1959999999999999e-06 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131365 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.805 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264464736999997, 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886024, 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131365 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.806 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.807 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498125+0000", 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.17e-06 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498129+0000", 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498131+0000", 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.666e-06 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.808 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500210+0000", 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020789559999999999 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500215+0000", 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528993+0000", 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028777174 2024-09-14T07:01:05.809 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529010+0000", 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7419999999999999e-05 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131366 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264446438999997, 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308831409999999, 2024-09-14T07:01:05.810 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131366 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498143+0000", 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498144+0000", 2024-09-14T07:01:05.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.474e-06 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498147+0000", 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.371e-06 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498150+0000", 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6410000000000002e-06 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500239+0000", 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002088842 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500243+0000", 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6689999999999999e-06 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529020+0000", 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287776420000001 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529025+0000", 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2440000000000001e-06 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131367 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264435378999998, 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030895135, 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.815 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131367 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498154+0000", 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2100000000000001e-07 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498155+0000", 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1310000000000001e-06 2024-09-14T07:01:05.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498157+0000", 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7870000000000002e-06 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498160+0000", 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9009999999999998e-06 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500264+0000", 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021031489999999999 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500268+0000", 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0929999999999996e-06 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529043+0000", 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287752139999999 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529048+0000", 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6389999999999997e-06 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131368 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264411246999998, 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308865389999999, 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.820 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131368 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.821 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498178+0000", 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8200000000000001e-07 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498179+0000", 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6929999999999999e-06 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498183+0000", 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9439999999999998e-06 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498188+0000", 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9409999999999998e-06 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500287+0000", 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020982269999999998 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500290+0000", 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6349999999999999e-06 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529059+0000", 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028768648 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529064+0000", 2024-09-14T07:01:05.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.169e-06 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131369 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264396054999999, 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886017, 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131369 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:05.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.28e-07 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498194+0000", 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1459999999999999e-06 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498197+0000", 2024-09-14T07:01:05.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0620000000000001e-06 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498200+0000", 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6000000000000001e-06 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500310+0000", 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021101840000000002 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500314+0000", 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.126e-06 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529074+0000", 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.02875961 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529079+0000", 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0610000000000003e-06 2024-09-14T07:01:05.829 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131370 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264380262000003, 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308930119999999, 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.830 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131370 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:01:05.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498209+0000", 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498210+0000", 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.187e-06 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498213+0000", 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.151e-06 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498216+0000", 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8289999999999999e-06 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500340+0000", 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021244229999999998 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500349+0000", 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529096+0000", 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287473309999999 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529101+0000", 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:01:05.834 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131371 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264361882000003, 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308901880000001, 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.835 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131371 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498229+0000", 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1110000000000002e-06 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498234+0000", 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:01:05.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498239+0000", 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.651e-06 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500382+0000", 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021424780000000002 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500389+0000", 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529112+0000", 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287229410000001 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529117+0000", 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9520000000000002e-06 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.839 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131372 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264340765999997, 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030884366, 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131372 2024-09-14T07:01:05.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8599999999999999e-07 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498251+0000", 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498257+0000", 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6370000000000002e-06 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498262+0000", 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500418+0000", 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021561800000000002 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500423+0000", 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1520000000000002e-06 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529127+0000", 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287049150000001 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529132+0000", 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.865e-06 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.844 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131373 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264307703, 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308673479999999, 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131373 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:01:05.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498283+0000", 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498290+0000", 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498295+0000", 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3759999999999999e-06 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500441+0000", 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021462849999999999 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500445+0000", 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8310000000000003e-06 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529142+0000", 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028697363 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529148+0000", 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.849 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131374 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264274086999997, 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308477090000001, 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131374 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:01:05.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498322+0000", 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5760000000000002e-06 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498328+0000", 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4609999999999996e-06 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498334+0000", 2024-09-14T07:01:05.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7130000000000002e-06 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500464+0000", 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002130617 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500468+0000", 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7340000000000002e-06 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529157+0000", 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286893669999999 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529162+0000", 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131375 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.854 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264233949999998, 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308296320000001, 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131375 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498358+0000", 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.317e-06 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498364+0000", 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5930000000000002e-06 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498369+0000", 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3630000000000002e-06 2024-09-14T07:01:05.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500488+0000", 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021187699999999999 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500492+0000", 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529179+0000", 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286873590000001 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529184+0000", 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131376 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.859 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264206411000004, 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030817713, 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131376 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498383+0000", 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498385+0000", 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.091e-06 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498391+0000", 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1070000000000001e-06 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498396+0000", 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4670000000000003e-06 2024-09-14T07:01:05.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500514+0000", 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021177449999999999 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500518+0000", 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8770000000000003e-06 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529195+0000", 2024-09-14T07:01:05.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028676938 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529200+0000", 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131377 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:05.864 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 49.264166885999998, 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308008749999999, 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131377 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:05.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498425+0000", 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5620000000000002e-06 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498431+0000", 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2539999999999998e-06 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498436+0000", 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.31e-06 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500539+0000", 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021025459999999998 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500543+0000", 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.597e-06 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529218+0000", 2024-09-14T07:01:05.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286753930000001 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529223+0000", 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.758e-06 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:05.869 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:05.871 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:01:05.871 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:05.870+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:05.971 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:05.970+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:06.016 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:06.016 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:06.016 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:06.025 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:01:06.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:01:06.108 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:06.107+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:06.168 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:06.168 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:06.168 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:06.168 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:06.169 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:06.173 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:06.172+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:06.177 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:01:06.187 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:06.188 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:06.205 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:06.205 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:06.221 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:06.237 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:06.253 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:06.254 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:06.272 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:06.272 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:06.274 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:06.273+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:06.283 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:06.283 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:06.288 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:06.288 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:06.292 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:01:06.295 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T07:01:06.313 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:06.314 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:06.375 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:06.374+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:06.406 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:06.407 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:06.415 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:01:06.476 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:06.474+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:06.520 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:06.520 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:06.528 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:01:06.578 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:06.577+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:06.672 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:06.672 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:06.678 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:06.677+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:06.681 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:01:06.779 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:06.778+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:06.785 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:06.785 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:06.794 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:01:06.876 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:01:06.876 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:01:06.876 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T07:01:06.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:01:06.879 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:06.877+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:06.897 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:06.898 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:06.906 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:01:06.990 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:06.989+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:07.052 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:07.052 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:07.061 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:01:07.105 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:07.104+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:07.170 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:07.170 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:07.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:01:07.181 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:07.180+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 110, 2024-09-14T07:01:07.185 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-14T06:58:59.479055+0000", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-14T07:01:06.287564+0000", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-14T06:59:11.718934+0000", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-14T07:00:26.509032+0000", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5806080, 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-14T07:01:07.186 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-14T07:01:07.187 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:07.187 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 6, 2024-09-14T07:01:07.187 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:01:07.187 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:01:07.187 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:01:07.187 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-14T07:01:07.187 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:11.765419+0000", 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-14T07:01:07.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-14T07:01:07.189 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "18", 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:01:07.190 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:01:07.191 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.192 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:01:07.193 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 6.9000000953674316, 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 6.9000000953674316, 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-14T07:01:07.194 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.82401084899902344, 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.93999999761581421, 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:18.343475+0000", 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:01:07.195 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-14T07:01:07.196 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "22", 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:01:07.197 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 22, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:01:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:01:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:01:07.200 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3.5, 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3.5, 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 3.5, 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 3.5, 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.82401084899902344, 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.93999999761581421, 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:27.257805+0000", 2024-09-14T07:01:07.201 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-14T07:01:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "110", 2024-09-14T07:01:07.203 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:01:07.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:01:07.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:01:07.204 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:01:07.204 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:01:07.204 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 36, 2024-09-14T07:01:07.204 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 110, 2024-09-14T07:01:07.204 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:01:07.204 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:01:07.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:01:07.206 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:01:07.207 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:01:07.207 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:01:07.207 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:01:07.207 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T07:01:07.207 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.309999942779541, 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.309999942779541, 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.309999942779541, 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.309999942779541, 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.82401084899902344, 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.93999999761581421, 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-09-14T07:01:07.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c036038d-cffb-4584-8851-69604a31ee40", 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.5920867919921875, 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:07.209 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 82, 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:07.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:07.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-14T07:01:07.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/2081065366", 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/2081065366", 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/2081065366", 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/2081065366", 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:07.214 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "1c80080f-932a-4957-b64b-f30c4bdb9340", 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 99, 2024-09-14T07:01:07.215 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:07.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:07.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.218 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:07.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-14T07:01:07.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:07.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:07.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6806/1688050909", 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6810/1688050909", 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/1688050909", 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/1688050909", 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:07.220 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f3544d11-8a0f-498c-a03c-5d34134c9773", 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 99, 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-14T07:01:07.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-14T07:01:07.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-14T07:01:07.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:07.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.226 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/1872259061", 2024-09-14T07:01:07.226 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/1872259061", 2024-09-14T07:01:07.226 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/1872259061", 2024-09-14T07:01:07.226 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/1872259061", 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d2261455-c91b-414b-925f-388f0e094dbe", 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.6944732666015625, 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:07.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 99, 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-14T07:01:07.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-14T07:01:07.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/1193301030", 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/1193301030", 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6811/1193301030", 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6807/1193301030", 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T07:01:07.232 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4088d23b-0d06-4415-9eba-6fb64355746c", 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 76, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6810", 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6811", 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.234 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6812", 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6813", 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.235 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6816", 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6817", 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6814", 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6815", 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6811/2283562461", 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6813/2283562461", 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6817/2283562461", 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6815/2283562461", 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "5e27cb53-ecce-4061-bc5a-be599bf18cc4", 2024-09-14T07:01:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 77, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6818", 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6819", 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6820", 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6821", 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6824", 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6825", 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:07.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6822", 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6823", 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6819/2212769626", 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6821/2212769626", 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6825/2212769626", 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6823/2212769626", 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T07:01:07.244 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ceb9a1f1-f54c-48d5-91d8-2287cac96ceb", 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:07.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6826", 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6827", 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6828", 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6829", 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6832", 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6833", 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6830", 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6831", 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:07.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6827/3197231389", 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6829/3197231389", 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6833/3197231389", 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6831/3197231389", 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85c59cd6-6e2b-4cc0-852d-d1f075a8826d", 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 99, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:07.251 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6800", 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6801", 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.252 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6802", 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6803", 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.253 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6806", 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6807", 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6804", 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6805", 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:07.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6801/2162534312", 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6803/2162534312", 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6807/2162534312", 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6805/2162534312", 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-14T07:01:07.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.064941+0000", 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:07.257 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.780598+0000", 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.127700+0000", 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:07.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.730995+0000", 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.864423+0000", 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:07.260 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.329361+0000", 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-14T07:01:07.261 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:10.107581+0000", 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.300043+0000", 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:07.262 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.8", 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:01:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.b", 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 7, 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1f", 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 5, 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 0 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-14T07:01:07.265 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 26, 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 10 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.267 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 26, 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-14T07:01:07.268 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-14T07:01:07.269 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-14T07:01:07.269 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-14T07:01:07.269 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:01:07.269 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:01:07.269 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-14T07:01:07.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:07.269 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:07.271 INFO:tasks.thrashosds.thrasher:j is {'epoch': 110, 'fsid': '32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6', 'created': '2024-09-14T06:58:59.479055+0000', 'modified': '2024-09-14T07:01:06.287564+0000', 'last_up_change': '2024-09-14T06:59:11.718934+0000', 'last_in_change': '2024-09-14T07:00:26.509032+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:59:11.765419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8240108489990234, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:59:18.343475+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.5, 'score_stable': 3.5, 'optimal_score': 1, 'raw_score_acting': 3.5, 'raw_score_stable': 3.5, 'primary_affinity_weighted': 0.8240108489990234, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:59:27.257805+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 110, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.8240108489990234, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}], 'osds': [{'osd': 0, 'uuid': 'c036038d-cffb-4584-8851-69604a31ee40', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5920867919921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 2081065366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 2081065366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 2081065366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 2081065366}]}, 'public_addr': '172.21.15.7:6817/2081065366', 'cluster_addr': '172.21.15.7:6819/2081065366', 'heartbeat_back_addr': '172.21.15.7:6823/2081065366', 'heartbeat_front_addr': '172.21.15.7:6821/2081065366', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1c80080f-932a-4957-b64b-f30c4bdb9340', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6806', 'nonce': 1688050909}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6810', 'nonce': 1688050909}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 1688050909}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 1688050909}]}, 'public_addr': '172.21.15.7:6806/1688050909', 'cluster_addr': '172.21.15.7:6810/1688050909', 'heartbeat_back_addr': '172.21.15.7:6815/1688050909', 'heartbeat_front_addr': '172.21.15.7:6813/1688050909', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f3544d11-8a0f-498c-a03c-5d34134c9773', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 1872259061}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 1872259061}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 1872259061}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 1872259061}]}, 'public_addr': '172.21.15.7:6825/1872259061', 'cluster_addr': '172.21.15.7:6827/1872259061', 'heartbeat_back_addr': '172.21.15.7:6831/1872259061', 'heartbeat_front_addr': '172.21.15.7:6829/1872259061', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd2261455-c91b-414b-925f-388f0e094dbe', 'up': 1, 'in': 1, 'weight': 0.6944732666015625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 1193301030}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 1193301030}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6809', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 1193301030}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6805', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 1193301030}]}, 'public_addr': '172.21.15.7:6801/1193301030', 'cluster_addr': '172.21.15.7:6803/1193301030', 'heartbeat_back_addr': '172.21.15.7:6811/1193301030', 'heartbeat_front_addr': '172.21.15.7:6807/1193301030', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4088d23b-0d06-4415-9eba-6fb64355746c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6810', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6811', 'nonce': 2283562461}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6812', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6813', 'nonce': 2283562461}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6816', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6817', 'nonce': 2283562461}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6814', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6815', 'nonce': 2283562461}]}, 'public_addr': '172.21.15.189:6811/2283562461', 'cluster_addr': '172.21.15.189:6813/2283562461', 'heartbeat_back_addr': '172.21.15.189:6817/2283562461', 'heartbeat_front_addr': '172.21.15.189:6815/2283562461', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5e27cb53-ecce-4061-bc5a-be599bf18cc4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6818', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6819', 'nonce': 2212769626}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6820', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6821', 'nonce': 2212769626}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6824', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6825', 'nonce': 2212769626}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6822', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6823', 'nonce': 2212769626}]}, 'public_addr': '172.21.15.189:6819/2212769626', 'cluster_addr': '172.21.15.189:6821/2212769626', 'heartbeat_back_addr': '172.21.15.189:6825/2212769626', 'heartbeat_front_addr': '172.21.15.189:6823/2212769626', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceb9a1f1-f54c-48d5-91d8-2287cac96ceb', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6826', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6827', 'nonce': 3197231389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6828', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6829', 'nonce': 3197231389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 3197231389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6830', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6831', 'nonce': 3197231389}]}, 'public_addr': '172.21.15.189:6827/3197231389', 'cluster_addr': '172.21.15.189:6829/3197231389', 'heartbeat_back_addr': '172.21.15.189:6833/3197231389', 'heartbeat_front_addr': '172.21.15.189:6831/3197231389', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '85c59cd6-6e2b-4cc0-852d-d1f075a8826d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6800', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6801', 'nonce': 2162534312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6802', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6803', 'nonce': 2162534312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6806', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6807', 'nonce': 2162534312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6804', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6805', 'nonce': 2162534312}]}, 'public_addr': '172.21.15.189:6801/2162534312', 'cluster_addr': '172.21.15.189:6803/2162534312', 'heartbeat_back_addr': '172.21.15.189:6807/2162534312', 'heartbeat_front_addr': '172.21.15.189:6805/2162534312', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.064941+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.780598+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.127700+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.730995+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.864423+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.329361+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T06:59:10.107581+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.300043+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.8', 'osds': [7, 0]}, {'pgid': '3.18', 'osds': [0, 5]}], 'pg_upmap_items': [{'pgid': '3.b', 'mappings': [{'from': 7, 'to': 5}]}, {'pgid': '3.1f', 'mappings': [{'from': 5, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 10}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 26, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:01:07.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:01:07.282 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:07.281+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:07.383 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:07.382+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:07.397 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:07.397 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:07.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:01:07.485 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:07.484+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:07.534 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:07.534 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:07.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:01:07.592 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:07.591+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:07.653 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:01:07.654 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:07.654 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:07.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:01:07.665 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-14T07:01:07.295755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":799309048,"num_objects":1910,"num_object_clones":503,"num_object_copies":3820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1910,"num_whiteouts":1021,"num_read":233505,"num_read_kb":524095,"num_write":21624,"num_write_kb":272977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2099,"num_bytes_recovered":84516687,"num_keys_recovered":235,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18124,"ondisk_log_size":18124,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":407888,"kb_used_data":38216,"kb_used_omap":10,"kb_used_meta":369461,"kb_avail":660194992,"statfs":{"total":676457349120,"available":676039671808,"internally_reserved":0,"allocated":39133184,"data_stored":259317076,"data_compressed":7325607,"data_compressed_allocated":19599360,"data_compressed_original":249693576,"omap_allocated":11157,"internal_metadata":378328171},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-72141274,"num_objects":-81,"num_object_clones":-15,"num_object_copies":-162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-81,"num_whiteouts":-13,"num_read":913,"num_read_kb":781,"num_write":164,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.635129"},"pg_stats":[{"pgid":"3.1f","version":"103'455","reported_seq":7478,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.652771+0000","last_change":"2024-09-14T07:01:01.653015+0000","last_active":"2024-09-14T07:01:01.652771+0000","last_peered":"2024-09-14T07:01:01.652771+0000","last_clean":"2024-09-14T07:01:01.652771+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:01:01.652771+0000","last_undegraded":"2024-09-14T07:01:01.652771+0000","last_fullsized":"2024-09-14T07:01:01.652771+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:36:52.077504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046844299999999997,"stat_sum":{"num_bytes":39058733,"num_objects":71,"num_object_clones":22,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":31,"num_read":7995,"num_read_kb":36967,"num_write":474,"num_write_kb":21653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.1e","version":"109'359","reported_seq":2684,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-14T07:01:05.294664+0000","last_change":"2024-09-14T07:01:05.294664+0000","last_active":"2024-09-14T07:01:05.294664+0000","last_peered":"2024-09-14T07:01:05.294664+0000","last_clean":"2024-09-14T07:01:05.294664+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:01:05.294664+0000","last_undegraded":"2024-09-14T07:01:05.294664+0000","last_fullsized":"2024-09-14T07:01:05.294664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":359,"log_dups_size":0,"ondisk_log_size":359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:18:24.909553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042608669999999998,"stat_sum":{"num_bytes":19078999,"num_objects":63,"num_object_clones":16,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":37,"num_read":2540,"num_read_kb":8250,"num_write":334,"num_write_kb":7992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"9"}]},{"pgid":"3.1d","version":"103'284","reported_seq":5547,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.667476+0000","last_change":"2024-09-14T07:01:01.667693+0000","last_active":"2024-09-14T07:01:01.667476+0000","last_peered":"2024-09-14T07:01:01.667476+0000","last_clean":"2024-09-14T07:01:01.667476+0000","last_became_active":"2024-09-14T07:00:56.618952+0000","last_became_peered":"2024-09-14T07:00:56.618952+0000","last_unstale":"2024-09-14T07:01:01.667476+0000","last_undegraded":"2024-09-14T07:01:01.667476+0000","last_fullsized":"2024-09-14T07:01:01.667476+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:21:23.966232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081010999999999995,"stat_sum":{"num_bytes":11183024,"num_objects":65,"num_object_clones":17,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":38,"num_read":5133,"num_read_kb":12828,"num_write":272,"num_write_kb":5161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.1c","version":"105'1953","reported_seq":4374,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.708511+0000","last_change":"2024-09-14T07:01:01.667521+0000","last_active":"2024-09-14T07:01:01.708511+0000","last_peered":"2024-09-14T07:01:01.708511+0000","last_clean":"2024-09-14T07:01:01.708511+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:01:01.708511+0000","last_undegraded":"2024-09-14T07:01:01.708511+0000","last_fullsized":"2024-09-14T07:01:01.708511+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1953,"log_dups_size":0,"ondisk_log_size":1953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:11:15.980672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066289400000000003,"stat_sum":{"num_bytes":11163318,"num_objects":65,"num_object_clones":19,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":33,"num_read":11426,"num_read_kb":22310,"num_write":3372,"num_write_kb":11214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.1b","version":"109'842","reported_seq":3125,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:01:06.297834+0000","last_change":"2024-09-14T07:01:06.298064+0000","last_active":"2024-09-14T07:01:06.297834+0000","last_peered":"2024-09-14T07:01:06.297834+0000","last_clean":"2024-09-14T07:01:06.297834+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:01:06.297834+0000","last_undegraded":"2024-09-14T07:01:06.297834+0000","last_fullsized":"2024-09-14T07:01:06.297834+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:55:59.715729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001214683,"stat_sum":{"num_bytes":37870794,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":4405,"num_read_kb":12207,"num_write":1114,"num_write_kb":20062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"a"}]},{"pgid":"3.1a","version":"103'1069","reported_seq":8164,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.754525+0000","last_change":"2024-09-14T07:01:01.667419+0000","last_active":"2024-09-14T07:01:01.754525+0000","last_peered":"2024-09-14T07:01:01.754525+0000","last_clean":"2024-09-14T07:01:01.754525+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:01:01.754525+0000","last_undegraded":"2024-09-14T07:01:01.754525+0000","last_fullsized":"2024-09-14T07:01:01.754525+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1069,"log_dups_size":0,"ondisk_log_size":1069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:31:30.881415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062082900000000004,"stat_sum":{"num_bytes":24913347,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":30,"num_read":12227,"num_read_kb":20348,"num_write":1558,"num_write_kb":6875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.19","version":"110'466","reported_seq":4758,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:01:06.298391+0000","last_change":"2024-09-14T07:01:06.298391+0000","last_active":"2024-09-14T07:01:06.298391+0000","last_peered":"2024-09-14T07:01:06.298391+0000","last_clean":"2024-09-14T07:01:06.298391+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:01:06.298391+0000","last_undegraded":"2024-09-14T07:01:06.298391+0000","last_fullsized":"2024-09-14T07:01:06.298391+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:36:02.615484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014794000000000001,"stat_sum":{"num_bytes":66486162,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":28,"num_read":4703,"num_read_kb":12466,"num_write":496,"num_write_kb":11826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"a"}]},{"pgid":"3.18","version":"104'552","reported_seq":3273,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.651353+0000","last_change":"2024-09-14T07:01:01.651600+0000","last_active":"2024-09-14T07:01:01.651353+0000","last_peered":"2024-09-14T07:01:01.651353+0000","last_clean":"2024-09-14T07:01:01.651353+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:01:01.651353+0000","last_undegraded":"2024-09-14T07:01:01.651353+0000","last_fullsized":"2024-09-14T07:01:01.651353+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:15:32.638956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046822500000000001,"stat_sum":{"num_bytes":12520679,"num_objects":60,"num_object_clones":16,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":31,"num_read":4483,"num_read_kb":9811,"num_write":612,"num_write_kb":6371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.17","version":"109'345","reported_seq":6443,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-14T07:01:05.366170+0000","last_change":"2024-09-14T07:01:05.298936+0000","last_active":"2024-09-14T07:01:05.366170+0000","last_peered":"2024-09-14T07:01:05.366170+0000","last_clean":"2024-09-14T07:01:05.366170+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:01:05.366170+0000","last_undegraded":"2024-09-14T07:01:05.366170+0000","last_fullsized":"2024-09-14T07:01:05.366170+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:39:03.227135+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0084501330000000003,"stat_sum":{"num_bytes":7821408,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":33,"num_read":7160,"num_read_kb":16389,"num_write":298,"num_write_kb":2840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"9"}]},{"pgid":"3.16","version":"104'381","reported_seq":4692,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.703863+0000","last_change":"2024-09-14T07:01:01.704042+0000","last_active":"2024-09-14T07:01:01.703863+0000","last_peered":"2024-09-14T07:01:01.703863+0000","last_clean":"2024-09-14T07:01:01.703863+0000","last_became_active":"2024-09-14T07:00:36.547481+0000","last_became_peered":"2024-09-14T07:00:36.547481+0000","last_unstale":"2024-09-14T07:01:01.703863+0000","last_undegraded":"2024-09-14T07:01:01.703863+0000","last_fullsized":"2024-09-14T07:01:01.703863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:42:27.003662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084920299999999998,"stat_sum":{"num_bytes":8745399,"num_objects":45,"num_object_clones":12,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":4902,"num_read_kb":8441,"num_write":345,"num_write_kb":3835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.15","version":"103'301","reported_seq":2270,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.649762+0000","last_change":"2024-09-14T07:01:01.650520+0000","last_active":"2024-09-14T07:01:01.649762+0000","last_peered":"2024-09-14T07:01:01.649762+0000","last_clean":"2024-09-14T07:01:01.649762+0000","last_became_active":"2024-09-14T07:00:56.618968+0000","last_became_peered":"2024-09-14T07:00:56.618968+0000","last_unstale":"2024-09-14T07:01:01.649762+0000","last_undegraded":"2024-09-14T07:01:01.649762+0000","last_fullsized":"2024-09-14T07:01:01.649762+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:47:06.317239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012627280000000001,"stat_sum":{"num_bytes":14202400,"num_objects":64,"num_object_clones":18,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":37,"num_read":1772,"num_read_kb":3897,"num_write":296,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.14","version":"103'700","reported_seq":5304,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.703632+0000","last_change":"2024-09-14T07:01:01.703806+0000","last_active":"2024-09-14T07:01:01.703632+0000","last_peered":"2024-09-14T07:01:01.703632+0000","last_clean":"2024-09-14T07:01:01.703632+0000","last_became_active":"2024-09-14T07:00:36.547583+0000","last_became_peered":"2024-09-14T07:00:36.547583+0000","last_unstale":"2024-09-14T07:01:01.703632+0000","last_undegraded":"2024-09-14T07:01:01.703632+0000","last_fullsized":"2024-09-14T07:01:01.703632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:59:38.973132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073154100000000005,"stat_sum":{"num_bytes":36856082,"num_objects":64,"num_object_clones":17,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":6497,"num_read_kb":12421,"num_write":1005,"num_write_kb":7155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.648704+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:01:01.648704+0000","last_peered":"2024-09-14T07:01:01.648704+0000","last_clean":"2024-09-14T07:01:01.648704+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:01:01.648704+0000","last_undegraded":"2024-09-14T07:01:01.648704+0000","last_fullsized":"2024-09-14T07:01:01.648704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"110'717","reported_seq":6646,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:01:06.301030+0000","last_change":"2024-09-14T07:01:06.301030+0000","last_active":"2024-09-14T07:01:06.301030+0000","last_peered":"2024-09-14T07:01:06.301030+0000","last_clean":"2024-09-14T07:01:06.301030+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:01:06.301030+0000","last_undegraded":"2024-09-14T07:01:06.301030+0000","last_fullsized":"2024-09-14T07:01:06.301030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:42.443806+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0038312760000000002,"stat_sum":{"num_bytes":15358049,"num_objects":49,"num_object_clones":10,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":29,"num_read":7780,"num_read_kb":8929,"num_write":853,"num_write_kb":5795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"a"}]},{"pgid":"2.6","version":"0'0","reported_seq":196,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:01:06.297375+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:01:06.297375+0000","last_peered":"2024-09-14T07:01:06.297375+0000","last_clean":"2024-09-14T07:01:06.297375+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:01:06.297375+0000","last_undegraded":"2024-09-14T07:01:06.297375+0000","last_fullsized":"2024-09-14T07:01:06.297375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"110'439","reported_seq":10976,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:01:06.378183+0000","last_change":"2024-09-14T07:01:06.298610+0000","last_active":"2024-09-14T07:01:06.378183+0000","last_peered":"2024-09-14T07:01:06.378183+0000","last_clean":"2024-09-14T07:01:06.378183+0000","last_became_active":"2024-09-14T06:59:51.751790+0000","last_became_peered":"2024-09-14T06:59:51.751790+0000","last_unstale":"2024-09-14T07:01:06.378183+0000","last_undegraded":"2024-09-14T07:01:06.378183+0000","last_fullsized":"2024-09-14T07:01:06.378183+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:07:25.616200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015733539999999999,"stat_sum":{"num_bytes":13269983,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":23,"num_read":12024,"num_read_kb":19539,"num_write":440,"num_write_kb":7938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"a"}]},{"pgid":"2.5","version":"0'0","reported_seq":186,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.649318+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:01:01.649318+0000","last_peered":"2024-09-14T07:01:01.649318+0000","last_clean":"2024-09-14T07:01:01.649318+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:01:01.649318+0000","last_undegraded":"2024-09-14T07:01:01.649318+0000","last_fullsized":"2024-09-14T07:01:01.649318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"109'461","reported_seq":3619,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-14T07:01:05.293907+0000","last_change":"2024-09-14T07:01:05.294653+0000","last_active":"2024-09-14T07:01:05.293907+0000","last_peered":"2024-09-14T07:01:05.293907+0000","last_clean":"2024-09-14T07:01:05.293907+0000","last_became_active":"2024-09-14T07:00:31.290234+0000","last_became_peered":"2024-09-14T07:00:31.290234+0000","last_unstale":"2024-09-14T07:01:05.293907+0000","last_undegraded":"2024-09-14T07:01:05.293907+0000","last_fullsized":"2024-09-14T07:01:05.293907+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:47.073213+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0039921369999999998,"stat_sum":{"num_bytes":20847224,"num_objects":45,"num_object_clones":10,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":29,"num_read":3801,"num_read_kb":4129,"num_write":406,"num_write_kb":4219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"9"}]},{"pgid":"2.4","version":"0'0","reported_seq":212,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-14T07:01:05.290414+0000","last_change":"2024-09-14T07:00:27.521374+0000","last_active":"2024-09-14T07:01:05.290414+0000","last_peered":"2024-09-14T07:01:05.290414+0000","last_clean":"2024-09-14T07:01:05.290414+0000","last_became_active":"2024-09-14T07:00:27.521101+0000","last_became_peered":"2024-09-14T07:00:27.521101+0000","last_unstale":"2024-09-14T07:01:05.290414+0000","last_undegraded":"2024-09-14T07:01:05.290414+0000","last_fullsized":"2024-09-14T07:01:05.290414+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:09:26.896693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"103'521","reported_seq":9914,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.743643+0000","last_change":"2024-09-14T07:01:01.703810+0000","last_active":"2024-09-14T07:01:01.743643+0000","last_peered":"2024-09-14T07:01:01.743643+0000","last_clean":"2024-09-14T07:01:01.743643+0000","last_became_active":"2024-09-14T07:00:31.290358+0000","last_became_peered":"2024-09-14T07:00:31.290358+0000","last_unstale":"2024-09-14T07:01:01.743643+0000","last_undegraded":"2024-09-14T07:01:01.743643+0000","last_fullsized":"2024-09-14T07:01:01.743643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":521,"log_dups_size":0,"ondisk_log_size":521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:47:26.976996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080240699999999999,"stat_sum":{"num_bytes":53043413,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":30,"num_read":8080,"num_read_kb":12871,"num_write":574,"num_write_kb":8318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.3","version":"105'642","reported_seq":8023,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.740729+0000","last_change":"2024-09-14T07:01:01.704878+0000","last_active":"2024-09-14T07:01:01.740729+0000","last_peered":"2024-09-14T07:01:01.740729+0000","last_clean":"2024-09-14T07:01:01.740729+0000","last_became_active":"2024-09-14T07:00:56.623321+0000","last_became_peered":"2024-09-14T07:00:56.623321+0000","last_unstale":"2024-09-14T07:01:01.740729+0000","last_undegraded":"2024-09-14T07:01:01.740729+0000","last_fullsized":"2024-09-14T07:01:01.740729+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:26.717500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013828130000000001,"stat_sum":{"num_bytes":20773075,"num_objects":56,"num_object_clones":13,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":30,"num_read":8608,"num_read_kb":10918,"num_write":867,"num_write_kb":6058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"2.2","version":"22'2","reported_seq":125,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.667142+0000","last_change":"2024-09-14T07:00:33.531473+0000","last_active":"2024-09-14T07:01:01.667142+0000","last_peered":"2024-09-14T07:01:01.667142+0000","last_clean":"2024-09-14T07:01:01.667142+0000","last_became_active":"2024-09-14T07:00:33.528631+0000","last_became_peered":"2024-09-14T07:00:33.528631+0000","last_unstale":"2024-09-14T07:01:01.667142+0000","last_undegraded":"2024-09-14T07:01:01.667142+0000","last_fullsized":"2024-09-14T07:01:01.667142+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:37:51.090265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"109'408","reported_seq":21845,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-14T07:01:05.294751+0000","last_change":"2024-09-14T07:01:05.295239+0000","last_active":"2024-09-14T07:01:05.294751+0000","last_peered":"2024-09-14T07:01:05.294751+0000","last_clean":"2024-09-14T07:01:05.294751+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:01:05.294751+0000","last_undegraded":"2024-09-14T07:01:05.294751+0000","last_fullsized":"2024-09-14T07:01:05.294751+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:32:32.158073+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.005070026,"stat_sum":{"num_bytes":16096419,"num_objects":56,"num_object_clones":17,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":25,"num_read":22134,"num_read_kb":44204,"num_write":451,"num_write_kb":17359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"9"}]},{"pgid":"2.1","version":"0'0","reported_seq":191,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T07:01:04.359516+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:01:04.359516+0000","last_peered":"2024-09-14T07:01:04.359516+0000","last_clean":"2024-09-14T07:01:04.359516+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:01:04.359516+0000","last_undegraded":"2024-09-14T07:01:04.359516+0000","last_fullsized":"2024-09-14T07:01:04.359516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"103'329","reported_seq":2144,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.703236+0000","last_change":"2024-09-14T07:01:01.703471+0000","last_active":"2024-09-14T07:01:01.703236+0000","last_peered":"2024-09-14T07:01:01.703236+0000","last_clean":"2024-09-14T07:01:01.703236+0000","last_became_active":"2024-09-14T07:00:30.290337+0000","last_became_peered":"2024-09-14T07:00:30.290337+0000","last_unstale":"2024-09-14T07:01:01.703236+0000","last_undegraded":"2024-09-14T07:01:01.703236+0000","last_fullsized":"2024-09-14T07:01:01.703236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:21:14.986216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056373999999999997,"stat_sum":{"num_bytes":11693770,"num_objects":57,"num_object_clones":16,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":35,"num_read":2614,"num_read_kb":12221,"num_write":289,"num_write_kb":7120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":186,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.648423+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T07:01:01.648423+0000","last_peered":"2024-09-14T07:01:01.648423+0000","last_clean":"2024-09-14T07:01:01.648423+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:01:01.648423+0000","last_undegraded":"2024-09-14T07:01:01.648423+0000","last_fullsized":"2024-09-14T07:01:01.648423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"110'653","reported_seq":10413,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T07:01:06.300804+0000","last_change":"2024-09-14T07:01:06.300804+0000","last_active":"2024-09-14T07:01:06.300804+0000","last_peered":"2024-09-14T07:01:06.300804+0000","last_clean":"2024-09-14T07:01:06.300804+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:01:06.300804+0000","last_undegraded":"2024-09-14T07:01:06.300804+0000","last_fullsized":"2024-09-14T07:01:06.300804+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:50.199284+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003960618,"stat_sum":{"num_bytes":41736873,"num_objects":56,"num_object_clones":9,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":30,"num_read":11372,"num_read_kb":36551,"num_write":882,"num_write_kb":12552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1a","length":"a"}]},{"pgid":"2.3","version":"20'1","reported_seq":207,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.648805+0000","last_change":"2024-09-14T07:00:27.523385+0000","last_active":"2024-09-14T07:01:01.648805+0000","last_peered":"2024-09-14T07:01:01.648805+0000","last_clean":"2024-09-14T07:01:01.648805+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:01:01.648805+0000","last_undegraded":"2024-09-14T07:01:01.648805+0000","last_fullsized":"2024-09-14T07:01:01.648805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:19.504174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":276,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.666659+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T07:01:01.666659+0000","last_peered":"2024-09-14T07:01:01.666659+0000","last_clean":"2024-09-14T07:01:01.666659+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:01:01.666659+0000","last_undegraded":"2024-09-14T07:01:01.666659+0000","last_fullsized":"2024-09-14T07:01:01.666659+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"106'524","reported_seq":9804,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T07:01:04.359977+0000","last_change":"2024-09-14T07:01:04.360207+0000","last_active":"2024-09-14T07:01:04.359977+0000","last_peered":"2024-09-14T07:01:04.359977+0000","last_clean":"2024-09-14T07:01:04.359977+0000","last_became_active":"2024-09-14T07:00:53.610102+0000","last_became_peered":"2024-09-14T07:00:53.610102+0000","last_unstale":"2024-09-14T07:01:04.359977+0000","last_undegraded":"2024-09-14T07:01:04.359977+0000","last_fullsized":"2024-09-14T07:01:04.359977+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:02.459701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000491743,"stat_sum":{"num_bytes":27867501,"num_objects":73,"num_object_clones":21,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":35,"num_read":9472,"num_read_kb":20927,"num_write":505,"num_write_kb":10482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4712244,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"8"}]},{"pgid":"3.9","version":"105'614","reported_seq":5003,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.654335+0000","last_change":"2024-09-14T07:01:01.654335+0000","last_active":"2024-09-14T07:01:01.654335+0000","last_peered":"2024-09-14T07:01:01.654335+0000","last_clean":"2024-09-14T07:01:01.654335+0000","last_became_active":"2024-09-14T06:59:28.804973+0000","last_became_peered":"2024-09-14T06:59:28.804973+0000","last_unstale":"2024-09-14T07:01:01.654335+0000","last_undegraded":"2024-09-14T07:01:01.654335+0000","last_fullsized":"2024-09-14T07:01:01.654335+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:39:31.798632+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0056371729999999997,"stat_sum":{"num_bytes":12056423,"num_objects":63,"num_object_clones":17,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":7396,"num_read_kb":17263,"num_write":704,"num_write_kb":7648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.a","version":"105'1020","reported_seq":8068,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.649663+0000","last_change":"2024-09-14T07:01:01.649663+0000","last_active":"2024-09-14T07:01:01.649663+0000","last_peered":"2024-09-14T07:01:01.649663+0000","last_clean":"2024-09-14T07:01:01.649663+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:01:01.649663+0000","last_undegraded":"2024-09-14T07:01:01.649663+0000","last_fullsized":"2024-09-14T07:01:01.649663+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1020,"log_dups_size":0,"ondisk_log_size":1020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:26:52.031094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000991956,"stat_sum":{"num_bytes":13919073,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":33,"num_read":10391,"num_read_kb":9647,"num_write":1466,"num_write_kb":3537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.b","version":"107'316","reported_seq":2683,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-14T07:01:05.290673+0000","last_change":"2024-09-14T07:01:05.291003+0000","last_active":"2024-09-14T07:01:05.290673+0000","last_peered":"2024-09-14T07:01:05.290673+0000","last_clean":"2024-09-14T07:01:05.290673+0000","last_became_active":"2024-09-14T07:00:13.450417+0000","last_became_peered":"2024-09-14T07:00:13.450417+0000","last_unstale":"2024-09-14T07:01:05.290673+0000","last_undegraded":"2024-09-14T07:01:05.290673+0000","last_fullsized":"2024-09-14T07:01:05.290673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":316,"log_dups_size":0,"ondisk_log_size":316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:31.878843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071933299999999995,"stat_sum":{"num_bytes":35749405,"num_objects":50,"num_object_clones":15,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":29,"num_read":1930,"num_read_kb":8359,"num_write":279,"num_write_kb":3165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"9"}]},{"pgid":"3.c","version":"108'410","reported_seq":5588,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T07:01:04.360831+0000","last_change":"2024-09-14T07:01:04.360831+0000","last_active":"2024-09-14T07:01:04.360831+0000","last_peered":"2024-09-14T07:01:04.360831+0000","last_clean":"2024-09-14T07:01:04.360831+0000","last_became_active":"2024-09-14T07:00:56.618738+0000","last_became_peered":"2024-09-14T07:00:56.618738+0000","last_unstale":"2024-09-14T07:01:04.360831+0000","last_undegraded":"2024-09-14T07:01:04.360831+0000","last_fullsized":"2024-09-14T07:01:04.360831+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:59:15.300981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011206969999999999,"stat_sum":{"num_bytes":28203797,"num_objects":62,"num_object_clones":20,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":32,"num_read":5146,"num_read_kb":11263,"num_write":407,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"8"}]},{"pgid":"3.d","version":"103'383","reported_seq":1967,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.650303+0000","last_change":"2024-09-14T07:01:01.651629+0000","last_active":"2024-09-14T07:01:01.650303+0000","last_peered":"2024-09-14T07:01:01.650303+0000","last_clean":"2024-09-14T07:01:01.650303+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:01:01.650303+0000","last_undegraded":"2024-09-14T07:01:01.650303+0000","last_fullsized":"2024-09-14T07:01:01.650303+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:53:56.768948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022968659999999998,"stat_sum":{"num_bytes":18093564,"num_objects":66,"num_object_clones":17,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":36,"num_read":1867,"num_read_kb":6569,"num_write":425,"num_write_kb":6923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.e","version":"103'381","reported_seq":12028,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.703918+0000","last_change":"2024-09-14T07:01:01.704090+0000","last_active":"2024-09-14T07:01:01.703918+0000","last_peered":"2024-09-14T07:01:01.703918+0000","last_clean":"2024-09-14T07:01:01.703918+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:01:01.703918+0000","last_undegraded":"2024-09-14T07:01:01.703918+0000","last_fullsized":"2024-09-14T07:01:01.703918+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:49:08.668726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065881399999999997,"stat_sum":{"num_bytes":47344559,"num_objects":65,"num_object_clones":19,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":36,"num_read":11913,"num_read_kb":54114,"num_write":397,"num_write_kb":18403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.f","version":"103'439","reported_seq":4488,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.754863+0000","last_change":"2024-09-14T07:01:01.653471+0000","last_active":"2024-09-14T07:01:01.754863+0000","last_peered":"2024-09-14T07:01:01.754863+0000","last_clean":"2024-09-14T07:01:01.754863+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:01:01.754863+0000","last_undegraded":"2024-09-14T07:01:01.754863+0000","last_fullsized":"2024-09-14T07:01:01.754863+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:40:36.445789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087275199999999997,"stat_sum":{"num_bytes":9330241,"num_objects":74,"num_object_clones":17,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":44,"num_read":5568,"num_read_kb":11277,"num_write":395,"num_write_kb":4659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.10","version":"103'386","reported_seq":3804,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.666860+0000","last_change":"2024-09-14T07:01:01.667100+0000","last_active":"2024-09-14T07:01:01.666860+0000","last_peered":"2024-09-14T07:01:01.666860+0000","last_clean":"2024-09-14T07:01:01.666860+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:01:01.666860+0000","last_undegraded":"2024-09-14T07:01:01.666860+0000","last_fullsized":"2024-09-14T07:01:01.666860+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:09.821675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050318699999999999,"stat_sum":{"num_bytes":16731653,"num_objects":57,"num_object_clones":16,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":30,"num_read":3592,"num_read_kb":12329,"num_write":381,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.11","version":"103'520","reported_seq":10619,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.651846+0000","last_change":"2024-09-14T07:01:01.651846+0000","last_active":"2024-09-14T07:01:01.651846+0000","last_peered":"2024-09-14T07:01:01.651846+0000","last_clean":"2024-09-14T07:01:01.651846+0000","last_became_active":"2024-09-14T07:00:31.290804+0000","last_became_peered":"2024-09-14T07:00:31.290804+0000","last_unstale":"2024-09-14T07:01:01.651846+0000","last_undegraded":"2024-09-14T07:01:01.651846+0000","last_fullsized":"2024-09-14T07:01:01.651846+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:28:17.361001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023372050000000002,"stat_sum":{"num_bytes":18956718,"num_objects":61,"num_object_clones":17,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":30,"num_read":11725,"num_read_kb":23724,"num_write":604,"num_write_kb":8965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"6"}]},{"pgid":"3.12","version":"107'668","reported_seq":6558,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T07:01:04.360685+0000","last_change":"2024-09-14T07:01:04.360868+0000","last_active":"2024-09-14T07:01:04.360685+0000","last_peered":"2024-09-14T07:01:04.360685+0000","last_clean":"2024-09-14T07:01:04.360685+0000","last_became_active":"2024-09-14T07:00:36.547241+0000","last_became_peered":"2024-09-14T07:00:36.547241+0000","last_unstale":"2024-09-14T07:01:04.360685+0000","last_undegraded":"2024-09-14T07:01:04.360685+0000","last_fullsized":"2024-09-14T07:01:04.360685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:24.489263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011575489999999999,"stat_sum":{"num_bytes":32792219,"num_objects":51,"num_object_clones":14,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":30,"num_read":8425,"num_read_kb":12984,"num_write":587,"num_write_kb":4981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1c","length":"8"}]},{"pgid":"3.13","version":"105'511","reported_seq":6032,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T07:01:01.770714+0000","last_change":"2024-09-14T07:01:01.649671+0000","last_active":"2024-09-14T07:01:01.770714+0000","last_peered":"2024-09-14T07:01:01.770714+0000","last_clean":"2024-09-14T07:01:01.770714+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:01:01.770714+0000","last_undegraded":"2024-09-14T07:01:01.770714+0000","last_fullsized":"2024-09-14T07:01:01.770714+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:18:14.249982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010875640000000001,"stat_sum":{"num_bytes":54954357,"num_objects":70,"num_object_clones":24,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":33,"num_read":6358,"num_read_kb":9887,"num_write":433,"num_write_kb":6945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1f","length":"6"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":798718661,"num_objects":1906,"num_object_clones":503,"num_object_copies":3812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1906,"num_whiteouts":1021,"num_read":233439,"num_read_kb":524040,"num_write":21521,"num_write_kb":271347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2096,"num_bytes_recovered":84516668,"num_keys_recovered":235,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30326784,"data_stored":254241639,"data_compressed":7346304,"data_compressed_allocated":19578880,"data_compressed_original":248739208,"omap_allocated":0,"internal_metadata":0},"log_size":18049,"ondisk_log_size":18049,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":0,"seq":64424509466,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57856,"kb_used_data":4452,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313984,"statfs":{"total":96636764160,"available":96577519616,"internally_reserved":0,"allocated":4558848,"data_stored":25237815,"data_compressed":691919,"data_compressed_allocated":1949696,"data_compressed_original":23972243,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59356,"kb_used_data":5940,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94312484,"statfs":{"total":96636764160,"available":96575983616,"internally_reserved":0,"allocated":6082560,"data_stored":36651835,"data_compressed":1106587,"data_compressed_allocated":3031040,"data_compressed_original":35129239,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81784,"kb_used_data":6176,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94290056,"statfs":{"total":96636764160,"available":96553017344,"internally_reserved":0,"allocated":6324224,"data_stored":51334534,"data_compressed":1431857,"data_compressed_allocated":3629056,"data_compressed_original":50043863,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71500,"kb_used_data":8356,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94300340,"statfs":{"total":96636764160,"available":96563548160,"internally_reserved":0,"allocated":8556544,"data_stored":50735300,"data_compressed":1840176,"data_compressed_allocated":4513792,"data_compressed_original":48580298,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36088,"kb_used_data":4896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335752,"statfs":{"total":96636764160,"available":96599810048,"internally_reserved":0,"allocated":5013504,"data_stored":37141020,"data_compressed":884910,"data_compressed_allocated":2445312,"data_compressed_original":35910665,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58352,"kb_used_data":4956,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94313488,"statfs":{"total":96636764160,"available":96577011712,"internally_reserved":0,"allocated":5074944,"data_stored":34859543,"data_compressed":798627,"data_compressed_allocated":2420736,"data_compressed_original":33641894,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42952,"kb_used_data":3440,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94328888,"statfs":{"total":96636764160,"available":96592781312,"internally_reserved":0,"allocated":3522560,"data_stored":23357029,"data_compressed":571531,"data_compressed_allocated":1609728,"data_compressed_original":22415374,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7315456,"data_stored":50152127,"data_compressed":1840176,"data_compressed_allocated":4513792,"data_compressed_original":48580298,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5021696,"data_stored":50160974,"data_compressed":1426726,"data_compressed_allocated":3592192,"data_compressed_original":49470423,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4841472,"data_stored":36068662,"data_compressed":1106587,"data_compressed_allocated":3031040,"data_compressed_original":35129239,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3325952,"data_stored":24660921,"data_compressed":691919,"data_compressed_allocated":1949696,"data_compressed_original":23972243,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3735552,"data_stored":34202257,"data_compressed":780688,"data_compressed_allocated":2396160,"data_compressed_original":33605030,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3661824,"data_stored":35931663,"data_compressed":879744,"data_compressed_allocated":2404352,"data_compressed_original":35324937,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":310283,"data_compressed":48933,"data_compressed_allocated":81920,"data_compressed_original":241664,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2240512,"data_stored":22754752,"data_compressed":571531,"data_compressed_allocated":1609728,"data_compressed_original":22415374,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:01:07.668 INFO:tasks.thrashosds.thrasher:Setting 3.11 to [6, 7] 2024-09-14T07:01:07.668 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.11', '6', '7'] 2024-09-14T07:01:07.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.11 6 7 2024-09-14T07:01:07.708 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:07.707+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:07.788 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:07.788+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:07.814 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:07.814 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:07.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:01:07.907 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:07.907+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:07.971 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:07.971 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:07.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:01:07.990 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:07.989+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:08.091 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:08.090+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:08.129 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:08.130 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:08.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:01:08.192 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:08.191+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:08.284 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:08.284 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:08.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:01:08.294 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:08.292+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:08.299 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.11 pg_upmap mapping to [6,7] 2024-09-14T07:01:08.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:01:08.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:01:08.404 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:08.404+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:08.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:01:08.507 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:08.507 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:08.506+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:08.525 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:08.525 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:08.542 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:08.542 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:08.559 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:08.559 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:08.576 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:08.576 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:08.593 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:08.593 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:08.596 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:08.595+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:08.597 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:08.597 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:08.597 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:08.597 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:08.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:01:08.610 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:08.611 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:08.627 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:08.697 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:08.697+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:08.718 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:08.718 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:08.718 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:08.718 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:08.718 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:08.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:01:08.810 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:08.810+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:08.875 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 109.12022375799999, 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:01:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:01:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:01:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:01:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.521811166000006, 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.516151523000005, 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:01:08.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:08.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.210315245000004, 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:01:08.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.210284453, 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:01:08.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:01:08.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.210261083000006, 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:01:08.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:01:08.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.209454574000006, 2024-09-14T07:01:08.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:08.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.209432180999997, 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:01:08.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.209412040999993, 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:01:08.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.209261885999993, 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.924 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:01:08.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.209206331999994, 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:01:08.928 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:01:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:01:08.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.209173290999999, 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:01:08.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:01:08.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:01:08.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:01:08.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:01:08.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.206820206000003, 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.206797640999994, 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:01:08.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:01:08.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.205940646000002, 2024-09-14T07:01:08.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:08.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:01:08.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.204723625, 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:08.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.204595784999995, 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.959 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.204575692000006, 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.960 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:01:08.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:01:08.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.203548799000004, 2024-09-14T07:01:08.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:01:08.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:01:08.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138497 3.14 3:298b17d3:::rbd_data.114f74bd3bbf.00000000000002d7:head [set-alloc-hint object_size 262144 write_size 262144,write 218355~43789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.556041516999997, 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97686345699999999, 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:08.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138497 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317983+0000", 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318003+0000", 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0610000000000001e-05 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318022+0000", 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8627e-05 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318030+0000", 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318101+0000", 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1162999999999996e-05 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318108+0000", 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:08.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291257+0000", 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97314914100000005 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291396+0000", 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013883799999999999 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291573+0000", 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017716800000000001 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.293147+0000", 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015740400000000001 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294753+0000", 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00160597 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294792+0000", 2024-09-14T07:01:08.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8758999999999999e-05 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294844+0000", 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2188000000000002e-05 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:08.975 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:08.977 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:01:08.977 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:08.976+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:09.077 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:09.076+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:09.087 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:09.087 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:09.087 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:09.087 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:09.096 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:01:09.178 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:09.177+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:09.200 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:09.200 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:09.200 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:09.200 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:09.200 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:09.208 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:01:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T07:01:09.293 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:09.291+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:09.365 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:09.365 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:09.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131358 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863687378999998, 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030805583, 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131358 2024-09-14T07:01:09.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498000+0000", 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498004+0000", 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2569999999999999e-06 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498007+0000", 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.507e-06 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499958+0000", 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001950989 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499962+0000", 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-14T07:01:09.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528792+0000", 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288306389999999 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528804+0000", 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2096000000000001e-05 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131359 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863667968999998, 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308363920000001, 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131359 2024-09-14T07:01:09.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498019+0000", 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498020+0000", 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4529999999999999e-06 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498023+0000", 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3450000000000002e-06 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498026+0000", 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.661e-06 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499995+0000", 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00196926 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500002+0000", 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6930000000000001e-06 2024-09-14T07:01:09.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528842+0000", 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288401300000001 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528855+0000", 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2595999999999999e-05 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131360 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863656966000001, 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030841943, 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131360 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498030+0000", 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:01:09.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498031+0000", 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1119999999999999e-06 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498034+0000", 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498036+0000", 2024-09-14T07:01:09.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.599e-06 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500030+0000", 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001993613 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500034+0000", 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1060000000000002e-06 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528867+0000", 2024-09-14T07:01:09.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288327429999999 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528871+0000", 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.673e-06 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131361 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:09.380 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863636221, 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308644520000001, 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131361 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:09.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498052+0000", 2024-09-14T07:01:09.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.24e-06 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498055+0000", 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1609999999999999e-06 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498058+0000", 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.05e-06 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500066+0000", 2024-09-14T07:01:09.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020081169999999998 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500075+0000", 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0559999999999994e-06 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528903+0000", 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288276810000001 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528915+0000", 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1938e-05 2024-09-14T07:01:09.384 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131362 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863617439999999, 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030860661, 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.385 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131362 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498075+0000", 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5639999999999996e-06 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498079+0000", 2024-09-14T07:01:09.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498081+0000", 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8660000000000002e-06 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500101+0000", 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020191219999999999 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500104+0000", 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6890000000000002e-06 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528925+0000", 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028820971 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528930+0000", 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3390000000000003e-06 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.389 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131363 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863601191999997, 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030874107, 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.390 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131363 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498086+0000", 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1009999999999999e-06 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498090+0000", 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5609999999999999e-06 2024-09-14T07:01:09.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498093+0000", 2024-09-14T07:01:09.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7360000000000001e-06 2024-09-14T07:01:09.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500136+0000", 2024-09-14T07:01:09.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020436030000000002 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500145+0000", 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9530000000000005e-06 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528948+0000", 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028803103 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528959+0000", 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0808999999999999e-05 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131364 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863584811000003, 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308734129999999, 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131364 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498102+0000", 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498103+0000", 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498107+0000", 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4089999999999999e-06 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498110+0000", 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7760000000000002e-06 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500177+0000", 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020678200000000002 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500183+0000", 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4979999999999997e-06 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528970+0000", 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287867049999999 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528975+0000", 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1959999999999999e-06 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131365 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.86356232, 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886024, 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.400 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131365 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498125+0000", 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.17e-06 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498129+0000", 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498131+0000", 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.666e-06 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500210+0000", 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020789559999999999 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500215+0000", 2024-09-14T07:01:09.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528993+0000", 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028777174 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529010+0000", 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7419999999999999e-05 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131366 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863544021999999, 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308831409999999, 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131366 2024-09-14T07:01:09.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498143+0000", 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498144+0000", 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.474e-06 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498147+0000", 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.371e-06 2024-09-14T07:01:09.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498150+0000", 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6410000000000002e-06 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500239+0000", 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002088842 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500243+0000", 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6689999999999999e-06 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529020+0000", 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287776420000001 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529025+0000", 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2440000000000001e-06 2024-09-14T07:01:09.409 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131367 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863532962000001, 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030895135, 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.410 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131367 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:01:09.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498154+0000", 2024-09-14T07:01:09.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2100000000000001e-07 2024-09-14T07:01:09.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498155+0000", 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1310000000000001e-06 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498157+0000", 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7870000000000002e-06 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498160+0000", 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9009999999999998e-06 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500264+0000", 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021031489999999999 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500268+0000", 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0929999999999996e-06 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529043+0000", 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287752139999999 2024-09-14T07:01:09.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529048+0000", 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6389999999999997e-06 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131368 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863508830000001, 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308865389999999, 2024-09-14T07:01:09.415 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131368 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498178+0000", 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8200000000000001e-07 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498179+0000", 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6929999999999999e-06 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498183+0000", 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9439999999999998e-06 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498188+0000", 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9409999999999998e-06 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500287+0000", 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020982269999999998 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500290+0000", 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6349999999999999e-06 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529059+0000", 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028768648 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529064+0000", 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.169e-06 2024-09-14T07:01:09.419 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131369 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863493638000001, 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886017, 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.420 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131369 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.28e-07 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498194+0000", 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1459999999999999e-06 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498197+0000", 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0620000000000001e-06 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498200+0000", 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6000000000000001e-06 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500310+0000", 2024-09-14T07:01:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021101840000000002 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500314+0000", 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.126e-06 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529074+0000", 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.02875961 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529079+0000", 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0610000000000003e-06 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131370 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863477844999998, 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308930119999999, 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.425 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131370 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:01:09.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498209+0000", 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498210+0000", 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.187e-06 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498213+0000", 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.151e-06 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498216+0000", 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8289999999999999e-06 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500340+0000", 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021244229999999998 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500349+0000", 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529096+0000", 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287473309999999 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529101+0000", 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.429 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131371 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863459464999998, 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308901880000001, 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131371 2024-09-14T07:01:09.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498229+0000", 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1110000000000002e-06 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498234+0000", 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498239+0000", 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.651e-06 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500382+0000", 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021424780000000002 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500389+0000", 2024-09-14T07:01:09.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529112+0000", 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287229410000001 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529117+0000", 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9520000000000002e-06 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131372 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863438348999999, 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030884366, 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131372 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.435 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:09.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8599999999999999e-07 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498251+0000", 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498257+0000", 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6370000000000002e-06 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498262+0000", 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500418+0000", 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021561800000000002 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500423+0000", 2024-09-14T07:01:09.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1520000000000002e-06 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529127+0000", 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287049150000001 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529132+0000", 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.865e-06 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131373 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863405286000003, 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308673479999999, 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131373 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.440 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:09.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498283+0000", 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498290+0000", 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498295+0000", 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3759999999999999e-06 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500441+0000", 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021462849999999999 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500445+0000", 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8310000000000003e-06 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529142+0000", 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028697363 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529148+0000", 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131374 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.444 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863371669999999, 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308477090000001, 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131374 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498322+0000", 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5760000000000002e-06 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498328+0000", 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4609999999999996e-06 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498334+0000", 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7130000000000002e-06 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500464+0000", 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002130617 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500468+0000", 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7340000000000002e-06 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529157+0000", 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286893669999999 2024-09-14T07:01:09.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529162+0000", 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131375 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863331533, 2024-09-14T07:01:09.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308296320000001, 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131375 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498358+0000", 2024-09-14T07:01:09.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.317e-06 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498364+0000", 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5930000000000002e-06 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498369+0000", 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3630000000000002e-06 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500488+0000", 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021187699999999999 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500492+0000", 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529179+0000", 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286873590000001 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529184+0000", 2024-09-14T07:01:09.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:01:09.454 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.454 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.454 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.454 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131376 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863303993999999, 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030817713, 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131376 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.455 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498383+0000", 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498385+0000", 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.091e-06 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498391+0000", 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1070000000000001e-06 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498396+0000", 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4670000000000003e-06 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500514+0000", 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021177449999999999 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500518+0000", 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8770000000000003e-06 2024-09-14T07:01:09.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529195+0000", 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028676938 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529200+0000", 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.459 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131377 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 52.863264469000001, 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308008749999999, 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131377 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498425+0000", 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5620000000000002e-06 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498431+0000", 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2539999999999998e-06 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498436+0000", 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.31e-06 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500539+0000", 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021025459999999998 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500543+0000", 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.597e-06 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529218+0000", 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286753930000001 2024-09-14T07:01:09.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529223+0000", 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.758e-06 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.464 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:09.466 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:01:09.466 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:09.464+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:09.568 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:09.566+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:09.610 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:09.610 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:09.610 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:09.610 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:09.618 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:01:09.669 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:09.667+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:09.721 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:09.721 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:09.721 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:09.721 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:09.722 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:09.730 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:01:09.773 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:09.769+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:09.871 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:09.871+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:09.876 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:09.876 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.709934824000001, 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.877 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:01:09.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.058591718999999, 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:01:09.881 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:01:09.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.058576250999998, 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.886 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:01:09.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:01:09.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.058564949999997, 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.890 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:01:09.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051821476999997, 2024-09-14T07:01:09.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:01:09.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:01:09.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051808872999999, 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.899 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:09.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:01:09.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:01:09.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:01:09.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:01:09.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:01:09.903 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051789356999997, 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.904 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.907 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051777315999999, 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.908 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.909 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:01:09.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:01:09.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051757014000003, 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:01:09.912 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:01:09.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:01:09.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051744698, 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:01:09.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:01:09.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.920 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051733257999999, 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.921 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.925 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051722759, 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051712565999999, 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:01:09.930 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:09.931 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.932 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.933 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051697019000002, 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.934 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:09.935 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:01:09.936 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.937 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051667410999997, 2024-09-14T07:01:09.938 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.939 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:01:09.940 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.941 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.942 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051645761000003, 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:01:09.943 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:09.944 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.945 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.946 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.051623522, 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:01:09.948 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.949 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.950 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.951 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.046027156999997, 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:09.952 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.953 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.954 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.955 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 62.046017104000001, 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.956 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.957 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.958 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.959 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:118426 3.b 3:d50c2793:::rbd_data.114f74d20483.000000000000001c:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 57.374635300999998, 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.95135023100000005, 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:09.960 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 118426 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.961 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500745+0000", 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500748+0000", 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1329999999999999e-06 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.962 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500759+0000", 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0475000000000001e-05 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500765+0000", 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500824+0000", 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.963 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500830+0000", 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5889999999999999e-06 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450787+0000", 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.94995644099999998 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450901+0000", 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114032 2024-09-14T07:01:09.964 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451052+0000", 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015113 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451746+0000", 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069442799999999997 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452006+0000", 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025975 2024-09-14T07:01:09.965 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452059+0000", 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3090000000000002e-05 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452095+0000", 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5395999999999998e-05 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:09.966 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:09.968 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:01:10.012 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:10.010+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:10.072 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:10.073 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:10.073 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:10.073 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:10.073 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:10.072+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:10.082 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:01:10.174 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:10.173+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:10.226 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:10.226 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:10.226 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:10.226 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:10.226 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:10.234 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:01:10.277 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:10.276+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:10.348 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:10.348 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:10.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 109.582673171, 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:01:10.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 91.509553905000004, 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:01:10.353 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:01:10.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:01:10.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:01:10.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:01:10.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:01:10.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:01:10.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:01:10.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 91.509297251999996, 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.360 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:01:10.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:01:10.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 91.483010178000001, 2024-09-14T07:01:10.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:01:10.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.368 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 91.482885564, 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.370 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:01:10.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 90.988266401999994, 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.373 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:01:10.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:01:10.377 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.713863742000001, 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:01:10.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:01:10.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:01:10.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.382 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:01:10.382 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:10.382 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 86.719248132000004, 2024-09-14T07:01:10.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:01:10.382 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.382 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.382 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.383 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:01:10.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:01:10.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:01:10.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 86.529075665999997, 2024-09-14T07:01:10.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:01:10.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 86.479942375999997, 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:01:10.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:01:10.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:01:10.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 86.111723412000003, 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:01:10.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.991154472999995, 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.400 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.404 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.860238592000002, 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.406 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:01:10.408 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.831201329999999, 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:01:10.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:01:10.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.805431908000003, 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:01:10.413 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:01:10.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:01:10.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.772073765000002, 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:01:10.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:01:10.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:01:10.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.607865984, 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:01:10.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:01:10.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.596486724000002, 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.426 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:01:10.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.430 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.596439029999999, 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:10.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 85.596096704999994, 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:01:10.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:01:10.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:01:10.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.439 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:10.441 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:01:10.441 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:10.440+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:10.542 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:10.541+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:10.585 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:10.585 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:10.585 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:10.585 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:10.594 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:01:10.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:01:10.642 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:10.642+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:10.697 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:10.697 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:10.697 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:10.697 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:10.697 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:10.706 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:01:10.750 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:10.750+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:10.772 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:10.788 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:10.805 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:10.805 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:10.812 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:10.812 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:10.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:10.812 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172493 3.8 3:11834377:::rbd_data.114fd4a72a77.0000000000000129:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.959061649999999, 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.395516846, 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.813 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172493 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:10.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.815 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.815 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.815 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851241+0000", 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164e-06 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851244+0000", 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1259999999999998e-06 2024-09-14T07:01:10.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851247+0000", 2024-09-14T07:01:10.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T07:01:10.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851660+0000", 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041335800000000001 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851667+0000", 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9809999999999997e-06 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597160+0000", 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549261700000002 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597167+0000", 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2629999999999997e-06 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246685+0000", 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649518327 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246756+0000", 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1210000000000004e-05 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172499 3.8 3:1336cd84:::rbd_data.114fd4a72a77.000000000000012f:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.959047343000002, 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3956566389999998, 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.821 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172499 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.22e-07 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851255+0000", 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.114e-06 2024-09-14T07:01:10.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851259+0000", 2024-09-14T07:01:10.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2909999999999999e-06 2024-09-14T07:01:10.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851261+0000", 2024-09-14T07:01:10.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.655e-06 2024-09-14T07:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851691+0000", 2024-09-14T07:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043023900000000001 2024-09-14T07:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.825 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851695+0000", 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4000000000000001e-06 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597190+0000", 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745495146 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597194+0000", 2024-09-14T07:01:10.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7050000000000001e-06 2024-09-14T07:01:10.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246801+0000", 2024-09-14T07:01:10.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649607445 2024-09-14T07:01:10.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246911+0000", 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000109422 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172507 3.8 3:17c8295f:::rbd_data.114fd4a72a77.0000000000000136:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.828 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.959036790999999, 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3957446349999998, 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172507 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851265+0000", 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.830 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851266+0000", 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851269+0000", 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0079999999999998e-06 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851271+0000", 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.531e-06 2024-09-14T07:01:10.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851725+0000", 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00045350300000000001 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851732+0000", 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9600000000000003e-06 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597223+0000", 2024-09-14T07:01:10.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745490926 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597230+0000", 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4499999999999998e-06 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246943+0000", 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649713363 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247009+0000", 2024-09-14T07:01:10.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5656999999999998e-05 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172606 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 14336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.959026583, 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.39623221, 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.834 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172606 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851276+0000", 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9399999999999993e-07 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851279+0000", 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0070000000000002e-06 2024-09-14T07:01:10.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851282+0000", 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9210000000000001e-06 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851756+0000", 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000474005 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851759+0000", 2024-09-14T07:01:10.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3670000000000001e-06 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597256+0000", 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549659599999996 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597260+0000", 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247047+0000", 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649786975 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247140+0000", 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3449999999999995e-05 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247507+0000", 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000366664 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.839 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172607 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 40960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.959009297000001, 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3965954819999999, 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172607 2024-09-14T07:01:10.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7000000000000001e-07 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851294+0000", 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3710000000000001e-06 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851297+0000", 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7850000000000002e-06 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851309+0000", 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1148000000000001e-05 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851788+0000", 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00047951899999999997 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851803+0000", 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5074e-05 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597290+0000", 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548693700000002 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597298+0000", 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3590000000000008e-06 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247532+0000", 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.650233759 2024-09-14T07:01:10.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247590+0000", 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8091000000000003e-05 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247887+0000", 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029716900000000001 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172608 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 67584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958980952000001, 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3970747800000001, 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172608 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851321+0000", 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0600000000000001e-07 2024-09-14T07:01:10.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851322+0000", 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.751e-06 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851331+0000", 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1799999999999996e-06 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851334+0000", 2024-09-14T07:01:10.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3409999999999999e-06 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851830+0000", 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00049571000000000001 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851833+0000", 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6069999999999999e-06 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597326+0000", 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549257700000005 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597333+0000", 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247917+0000", 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6505841430000001 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248020+0000", 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102581 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248395+0000", 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037526200000000001 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172609 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 94208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.851 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958961305999999, 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3975014290000001, 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172609 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9200000000000002e-07 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851342+0000", 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.451e-06 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851346+0000", 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9999999999999998e-06 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851348+0000", 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5780000000000001e-06 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851868+0000", 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051942399999999999 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851876+0000", 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6850000000000007e-06 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597371+0000", 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745494502 2024-09-14T07:01:10.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597378+0000", 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6129999999999998e-06 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248428+0000", 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.651049502 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248482+0000", 2024-09-14T07:01:10.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3767999999999997e-05 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248841+0000", 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00035961400000000002 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172610 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 120832~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:10.857 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958940502000001, 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3980250019999998, 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172610 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:10.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6800000000000002e-07 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851362+0000", 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4300000000000001e-06 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851366+0000", 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.811e-06 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851369+0000", 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4739999999999999e-06 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851901+0000", 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053255399999999997 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851905+0000", 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4170000000000001e-06 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597404+0000", 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549902099999998 2024-09-14T07:01:10.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597408+0000", 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.527e-06 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248892+0000", 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65148359 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249005+0000", 2024-09-14T07:01:10.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113397 2024-09-14T07:01:10.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249386+0000", 2024-09-14T07:01:10.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038051300000000002 2024-09-14T07:01:10.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.863 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.863 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172611 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 147456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958922223999998, 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3985748930000002, 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.864 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172611 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.79e-07 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851381+0000", 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7379999999999999e-06 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851385+0000", 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5750000000000001e-06 2024-09-14T07:01:10.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851387+0000", 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851934+0000", 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00054657299999999996 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851942+0000", 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2549999999999997e-06 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597437+0000", 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549549800000003 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597445+0000", 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5839999999999997e-06 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249454+0000", 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6520086940000001 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249576+0000", 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012215899999999999 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249954+0000", 2024-09-14T07:01:10.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037802300000000001 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172612 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 174080~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958910627000002, 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3988136949999999, 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.870 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172612 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851392+0000", 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851395+0000", 2024-09-14T07:01:10.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2449999999999998e-06 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851398+0000", 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4660000000000002e-06 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851978+0000", 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058031599999999999 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851984+0000", 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1319999999999999e-06 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597469+0000", 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548469799999995 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597473+0000", 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8510000000000001e-06 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249990+0000", 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6525175509999999 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250091+0000", 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010111900000000001 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250204+0000", 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113034 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.875 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172613 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 200704~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958900654000001, 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3989707839999999, 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172613 2024-09-14T07:01:10.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:10.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.05e-07 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851402+0000", 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.026e-06 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851405+0000", 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2440000000000002e-06 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851407+0000", 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3920000000000001e-06 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852016+0000", 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060825699999999996 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852024+0000", 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0639999999999994e-06 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597505+0000", 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548078699999998 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597513+0000", 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1659999999999994e-06 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250226+0000", 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652713648 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250279+0000", 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2880999999999997e-05 2024-09-14T07:01:10.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250371+0000", 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2114000000000006e-05 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172614 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 227328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958890843999999, 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3991723779999998, 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.881 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172614 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:10.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851411+0000", 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851412+0000", 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4900000000000004e-07 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851415+0000", 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.942e-06 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851417+0000", 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000002e-06 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852054+0000", 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00063717699999999995 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852058+0000", 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6739999999999999e-06 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597543+0000", 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548541400000001 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597548+0000", 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6140000000000002e-06 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250407+0000", 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652859388 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250487+0000", 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9523000000000004e-05 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250583+0000", 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6026999999999994e-05 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172615 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 253952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.886 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958881184999999, 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3993917979999999, 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172615 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:10.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851421+0000", 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669999999999999e-06 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851425+0000", 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0110000000000002e-06 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851428+0000", 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.452e-06 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852092+0000", 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000664497 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852099+0000", 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1199999999999996e-06 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597585+0000", 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548563599999995 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597593+0000", 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7989999999999994e-06 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250631+0000", 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653038477 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250689+0000", 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7192999999999999e-05 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250812+0000", 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000123344 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172719 3.8 3:10ac98f7:::rbd_data.114fd4a72a77.000000000000018d:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958870369, 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3995141609999999, 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.892 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172719 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851432+0000", 2024-09-14T07:01:10.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851435+0000", 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1980000000000001e-06 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851442+0000", 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9419999999999998e-06 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852133+0000", 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069105100000000004 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852139+0000", 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597622+0000", 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548300599999995 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597635+0000", 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3183000000000001e-05 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250847+0000", 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6532118730000001 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250945+0000", 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7770999999999996e-05 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172725 3.8 3:16c5e555:::rbd_data.114fd4a72a77.0000000000000193:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958843730000002, 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3996608569999998, 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.898 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172725 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851459+0000", 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.626e-06 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851466+0000", 2024-09-14T07:01:10.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8349999999999998e-06 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851469+0000", 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852192+0000", 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072263799999999995 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852201+0000", 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2129999999999997e-06 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597665+0000", 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74546438299999995 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597673+0000", 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6410000000000002e-06 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.902 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251000+0000", 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653327011 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251118+0000", 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000118446 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172726 3.8 3:143e5002:::rbd_data.114fd4a72a77.0000000000000194:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.903 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958828475000001, 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3998113490000001, 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172726 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.904 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.905 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851474+0000", 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2139999999999999e-06 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851477+0000", 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.208e-06 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851480+0000", 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4700000000000001e-06 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.906 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852248+0000", 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00076849899999999996 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852257+0000", 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6840000000000002e-06 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597699+0000", 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74544160400000004 2024-09-14T07:01:10.907 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597706+0000", 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4170000000000003e-06 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251171+0000", 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65346452 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251284+0000", 2024-09-14T07:01:10.908 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113498 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172760 3.8 3:16edf585:::rbd_data.114fd4a72a77.00000000000001b6:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958805449, 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3999111539999998, 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.909 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172760 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.910 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851497+0000", 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1209999999999999e-06 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851501+0000", 2024-09-14T07:01:10.911 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3279999999999999e-06 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851505+0000", 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.275e-06 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852297+0000", 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079227500000000005 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852306+0000", 2024-09-14T07:01:10.912 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6659999999999997e-06 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597736+0000", 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74543018500000002 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597743+0000", 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4290000000000004e-06 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.913 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251309+0000", 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6535660569999999 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251407+0000", 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7603999999999996e-05 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172775 3.8 3:104e7293:::rbd_data.114fd4a72a77.00000000000001c5:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:10.914 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958777378000001, 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4000188580000001, 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172775 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.915 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6100000000000002e-07 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851531+0000", 2024-09-14T07:01:10.916 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2760000000000003e-06 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851536+0000", 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2640000000000002e-06 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851538+0000", 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5289999999999998e-06 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852358+0000", 2024-09-14T07:01:10.917 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081989599999999997 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852368+0000", 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597767+0000", 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74539930200000004 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.918 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597771+0000", 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4039999999999997e-06 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251441+0000", 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6536695479999999 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251543+0000", 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000101919 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.919 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172778 3.8 3:12efabcb:::rbd_data.114fd4a72a77.00000000000001c8:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958759267000001, 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400145024, 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172778 2024-09-14T07:01:10.920 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:10.921 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3900000000000001e-07 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851543+0000", 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851546+0000", 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851549+0000", 2024-09-14T07:01:10.922 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4530000000000001e-06 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852399+0000", 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00085052200000000004 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852407+0000", 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.238e-06 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.923 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597814+0000", 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74540710700000001 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597821+0000", 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251576+0000", 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653755171 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.924 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251687+0000", 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011061 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172814 3.8 3:166e6639:::rbd_data.114fd4a72a77.00000000000001ec:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 20.958749262000001, 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400279984, 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:10.925 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172814 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:10.926 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592075+0000", 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74052290899999995 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592081+0000", 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6339999999999998e-06 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592090+0000", 2024-09-14T07:01:10.927 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5929999999999995e-06 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592097+0000", 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4930000000000003e-06 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597854+0000", 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0057574890000000002 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597858+0000", 2024-09-14T07:01:10.928 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.228e-06 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251725+0000", 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6538667419999999 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251832+0000", 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000106896 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.929 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:10.930 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:10.930 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:10.930 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:10.932 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:10.932 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:10.930+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:10.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:01:10.933 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:10.933 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:10.934 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:11.031 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:11.031+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:11.082 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:11.082 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:11.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:01:11.133 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:11.133+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:11.197 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:11.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:01:11.247 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:11.246+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:11.311 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:01:11.335 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:11.334+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:11.394 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:11.394 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:11.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:01:11.452 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:11.451+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:11.529 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:11.529 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:11.536 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:11.535+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:11.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:01:11.637 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:11.636+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:11.686 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:11.686 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:11.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:01:11.738 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:11.737+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:11.839 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:11.838+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:11.842 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:11.842 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:11.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:01:11.939 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:11.939+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:11.999 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:11.999 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:12.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:01:12.049 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:12.048+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:12.142 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:12.141+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:12.155 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:12.156 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:12.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:01:12.243 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:12.243+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:12.312 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:12.312 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:12.323 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T07:01:12.326 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:01:12.344 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:12.344+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:12.445 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:12.445+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:12.447 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:12.447 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:12.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:01:12.546 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:12.545+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:12.569 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:12.569 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:12.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:01:12.647 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:12.646+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:12.685 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:12.685 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:12.694 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:01:12.748 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:12.747+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:12.805 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:12.805 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:12.814 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:01:12.856 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:12.855+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:12.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:01:12.957 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:12.958 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:12.966 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:01:12.980 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:12.979+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:13.051 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:13.050+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:13.060 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:13.061 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:13.070 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:13.070 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:13.077 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:13.079 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:01:13.094 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:13.094 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:13.110 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:13.127 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:13.127 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:13.143 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:13.144 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:13.152 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:13.151+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:13.160 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:13.160 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:13.177 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:13.177 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:13.184 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:13.184 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:13.184 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:13.184 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:13.193 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:01:13.253 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:13.252+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:13.321 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:01:13.321 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:01:13.322 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-14T07:01:13.322 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T07:01:13.322 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T07:01:13.322 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-14T07:01:13.341 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:13.341 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:13.341 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:13.341 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:13.341 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:13.350 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:01:13.391 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:13.390+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:13.434 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:13.435 INFO:teuthology.orchestra.run.smithi189.stdout: "success": "" 2024-09-14T07:01:13.435 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:13.453 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:13.453+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:13.499 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.332682970999997, 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.500 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:01:13.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:01:13.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.681339866000002, 2024-09-14T07:01:13.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:01:13.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:01:13.508 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.681324398000001, 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.509 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.681313097, 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.513 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:01:13.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674569624, 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:01:13.521 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674557019999995, 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.522 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.526 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.527 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:01:13.527 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674537504, 2024-09-14T07:01:13.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:01:13.527 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.527 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.527 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.527 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.530 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.531 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674525462999995, 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:01:13.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.535 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674505160999999, 2024-09-14T07:01:13.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:13.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:01:13.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:01:13.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674492845000003, 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.541 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674481404999995, 2024-09-14T07:01:13.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.549 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674470905999996, 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:01:13.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:01:13.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674460713000002, 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.554 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:13.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:01:13.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:01:13.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:01:13.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674445165999998, 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:01:13.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:01:13.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674415558000007, 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.563 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674393907999999, 2024-09-14T07:01:13.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:01:13.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.571 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.674371668999996, 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:01:13.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.668775303999993, 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.576 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:01:13.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 65.668765250999996, 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:01:13.580 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:01:13.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:118426 3.b 3:d50c2793:::rbd_data.114f74d20483.000000000000001c:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 60.997383448000001, 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.95135023100000005, 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 118426 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:13.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500745+0000", 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500748+0000", 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1329999999999999e-06 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500759+0000", 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0475000000000001e-05 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500765+0000", 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.587 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500824+0000", 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500830+0000", 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5889999999999999e-06 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450787+0000", 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.94995644099999998 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:13.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450901+0000", 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114032 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451052+0000", 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015113 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451746+0000", 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069442799999999997 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:13.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452006+0000", 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025975 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452059+0000", 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3090000000000002e-05 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452095+0000", 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5395999999999998e-05 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:13.590 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:13.591 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:13.592 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:01:13.592 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:13.591+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:13.692 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:13.692+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:13.735 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:13.735 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:13.735 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:13.735 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:13.744 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:01:13.794 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:13.793+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:13.888 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:13.888 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:13.888 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:13.888 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:13.888 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:13.895 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:13.894+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:13.897 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:01:13.996 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:13.995+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:14.001 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:14.001 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:14.001 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 113.236200314, 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:01:14.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 95.163081047999995, 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.006 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.007 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:01:14.008 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 95.162824395000001, 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.012 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:01:14.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:01:14.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.016 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 95.136537321000006, 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:01:14.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 95.136412707000005, 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:01:14.021 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:01:14.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:01:14.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 94.641793544999999, 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:01:14.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.029 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.367390885000006, 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:01:14.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:01:14.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:14.035 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 90.372775274999995, 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:14.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:01:14.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.039 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 90.182602809000002, 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.041 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 90.133469519000002, 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:01:14.044 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:01:14.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:01:14.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.765250554999994, 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:01:14.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:01:14.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.644681616, 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.053 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.057 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.513765735000007, 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:01:14.060 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.061 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.484728473000004, 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.062 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:01:14.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.066 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.458959050999994, 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:01:14.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:01:14.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.425600908000007, 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.071 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:01:14.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.073 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.075 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.261393127000005, 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:01:14.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.250013867000007, 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.080 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.081 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.082 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.083 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:01:14.084 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.249966173000004, 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.085 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.086 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:01:14.087 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:01:14.088 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 89.249623847999999, 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.089 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:14.090 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:01:14.091 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:14.092 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.093 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:14.094 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:01:14.097 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:14.096+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:14.198 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:14.197+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:14.240 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:14.240 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:14.240 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:14.240 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:14.248 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:01:14.332 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:14.332+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:14.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:01:14.399 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:14.399+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:14.400 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:14.400 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:14.400 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:14.400 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:14.400 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:14.409 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:01:14.500 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:14.499+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:14.556 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172493 3.8 3:11834377:::rbd_data.114fd4a72a77.0000000000000129:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703611726999998, 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.395516846, 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.557 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172493 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851241+0000", 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164e-06 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851244+0000", 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1259999999999998e-06 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851247+0000", 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851660+0000", 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041335800000000001 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851667+0000", 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9809999999999997e-06 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597160+0000", 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549261700000002 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597167+0000", 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2629999999999997e-06 2024-09-14T07:01:14.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246685+0000", 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649518327 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246756+0000", 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1210000000000004e-05 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172499 3.8 3:1336cd84:::rbd_data.114fd4a72a77.000000000000012f:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703597420000001, 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3956566389999998, 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172499 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.563 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:14.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.22e-07 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851255+0000", 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.114e-06 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851259+0000", 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2909999999999999e-06 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851261+0000", 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.655e-06 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851691+0000", 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043023900000000001 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851695+0000", 2024-09-14T07:01:14.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4000000000000001e-06 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597190+0000", 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745495146 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597194+0000", 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7050000000000001e-06 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246801+0000", 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649607445 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246911+0000", 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000109422 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172507 3.8 3:17c8295f:::rbd_data.114fd4a72a77.0000000000000136:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703586867999999, 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3957446349999998, 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172507 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.569 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851265+0000", 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851266+0000", 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851269+0000", 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0079999999999998e-06 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851271+0000", 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.531e-06 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851725+0000", 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00045350300000000001 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851732+0000", 2024-09-14T07:01:14.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9600000000000003e-06 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597223+0000", 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745490926 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597230+0000", 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4499999999999998e-06 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246943+0000", 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649713363 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247009+0000", 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5656999999999998e-05 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172606 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 14336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.574 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.70357666, 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.39623221, 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172606 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851276+0000", 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9399999999999993e-07 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851279+0000", 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0070000000000002e-06 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851282+0000", 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9210000000000001e-06 2024-09-14T07:01:14.577 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851756+0000", 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000474005 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851759+0000", 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3670000000000001e-06 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.578 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597256+0000", 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549659599999996 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597260+0000", 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247047+0000", 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649786975 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.579 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247140+0000", 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3449999999999995e-05 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247507+0000", 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000366664 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.580 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172607 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 40960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703559374000001, 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3965954819999999, 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172607 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.581 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:14.582 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7000000000000001e-07 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851294+0000", 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3710000000000001e-06 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851297+0000", 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7850000000000002e-06 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.583 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851309+0000", 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1148000000000001e-05 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851788+0000", 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00047951899999999997 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851803+0000", 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5074e-05 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.584 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597290+0000", 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548693700000002 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597298+0000", 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3590000000000008e-06 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247532+0000", 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.650233759 2024-09-14T07:01:14.585 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247590+0000", 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8091000000000003e-05 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247887+0000", 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029716900000000001 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.586 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172608 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 67584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703531029000001, 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3970747800000001, 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172608 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.587 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851321+0000", 2024-09-14T07:01:14.588 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0600000000000001e-07 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851322+0000", 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.751e-06 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851331+0000", 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1799999999999996e-06 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.589 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851334+0000", 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3409999999999999e-06 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851830+0000", 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00049571000000000001 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851833+0000", 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6069999999999999e-06 2024-09-14T07:01:14.590 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597326+0000", 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549257700000005 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597333+0000", 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.591 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247917+0000", 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6505841430000001 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248020+0000", 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102581 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248395+0000", 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037526200000000001 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.592 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172609 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 94208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703511382999999, 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3975014290000001, 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172609 2024-09-14T07:01:14.593 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.594 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9200000000000002e-07 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851342+0000", 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.451e-06 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851346+0000", 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9999999999999998e-06 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.595 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851348+0000", 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5780000000000001e-06 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851868+0000", 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051942399999999999 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851876+0000", 2024-09-14T07:01:14.596 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6850000000000007e-06 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597371+0000", 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745494502 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597378+0000", 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6129999999999998e-06 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.597 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248428+0000", 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.651049502 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248482+0000", 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3767999999999997e-05 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248841+0000", 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00035961400000000002 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.598 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172610 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 120832~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703490579, 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3980250019999998, 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172610 2024-09-14T07:01:14.599 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.600 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6800000000000002e-07 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851362+0000", 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4300000000000001e-06 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851366+0000", 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.811e-06 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.601 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851369+0000", 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4739999999999999e-06 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851901+0000", 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053255399999999997 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851905+0000", 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4170000000000001e-06 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.602 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597404+0000", 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549902099999998 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597408+0000", 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.527e-06 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248892+0000", 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65148359 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.603 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249005+0000", 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113397 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249386+0000", 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038051300000000002 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172611 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 147456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.604 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703472301000001, 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3985748930000002, 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172611 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.605 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.79e-07 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.606 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851381+0000", 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7379999999999999e-06 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851385+0000", 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5750000000000001e-06 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851387+0000", 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.607 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851934+0000", 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00054657299999999996 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851942+0000", 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2549999999999997e-06 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597437+0000", 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549549800000003 2024-09-14T07:01:14.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597445+0000", 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5839999999999997e-06 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249454+0000", 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6520086940000001 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.609 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249576+0000", 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012215899999999999 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249954+0000", 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037802300000000001 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172612 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 174080~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:14.610 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703460704000001, 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3988136949999999, 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172612 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:14.611 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.612 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851392+0000", 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851395+0000", 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2449999999999998e-06 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851398+0000", 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4660000000000002e-06 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.613 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851978+0000", 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058031599999999999 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851984+0000", 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1319999999999999e-06 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597469+0000", 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548469799999995 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.614 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597473+0000", 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8510000000000001e-06 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249990+0000", 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6525175509999999 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250091+0000", 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010111900000000001 2024-09-14T07:01:14.615 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250204+0000", 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113034 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172613 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 200704~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703450731, 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3989707839999999, 2024-09-14T07:01:14.616 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172613 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.617 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.05e-07 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851402+0000", 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.026e-06 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.618 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851405+0000", 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2440000000000002e-06 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851407+0000", 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3920000000000001e-06 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852016+0000", 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060825699999999996 2024-09-14T07:01:14.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852024+0000", 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0639999999999994e-06 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597505+0000", 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548078699999998 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597513+0000", 2024-09-14T07:01:14.620 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1659999999999994e-06 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250226+0000", 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652713648 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250279+0000", 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2880999999999997e-05 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.621 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250371+0000", 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2114000000000006e-05 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172614 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 227328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703440920999999, 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3991723779999998, 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.622 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172614 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.623 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851411+0000", 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851412+0000", 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4900000000000004e-07 2024-09-14T07:01:14.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851415+0000", 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.942e-06 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851417+0000", 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000002e-06 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.625 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852054+0000", 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00063717699999999995 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852058+0000", 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6739999999999999e-06 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597543+0000", 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548541400000001 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.626 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597548+0000", 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6140000000000002e-06 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250407+0000", 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652859388 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.627 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250487+0000", 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9523000000000004e-05 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250583+0000", 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6026999999999994e-05 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.628 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172615 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 253952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703431261999999, 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3993917979999999, 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172615 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.629 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.630 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851421+0000", 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669999999999999e-06 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851425+0000", 2024-09-14T07:01:14.631 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0110000000000002e-06 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851428+0000", 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.452e-06 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852092+0000", 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000664497 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.632 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852099+0000", 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1199999999999996e-06 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597585+0000", 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548563599999995 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597593+0000", 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7989999999999994e-06 2024-09-14T07:01:14.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250631+0000", 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653038477 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250689+0000", 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7192999999999999e-05 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.634 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250812+0000", 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000123344 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172719 3.8 3:10ac98f7:::rbd_data.114fd4a72a77.000000000000018d:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703420445999999, 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3995141609999999, 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.635 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172719 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.636 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851432+0000", 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:01:14.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851435+0000", 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1980000000000001e-06 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851442+0000", 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9419999999999998e-06 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852133+0000", 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069105100000000004 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.638 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852139+0000", 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597622+0000", 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548300599999995 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597635+0000", 2024-09-14T07:01:14.639 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3183000000000001e-05 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250847+0000", 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6532118730000001 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250945+0000", 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7770999999999996e-05 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.640 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172725 3.8 3:16c5e555:::rbd_data.114fd4a72a77.0000000000000193:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703393807000001, 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3996608569999998, 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172725 2024-09-14T07:01:14.641 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:14.642 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851459+0000", 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.626e-06 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851466+0000", 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8349999999999998e-06 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.643 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851469+0000", 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852192+0000", 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072263799999999995 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852201+0000", 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2129999999999997e-06 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.644 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597665+0000", 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74546438299999995 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597673+0000", 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6410000000000002e-06 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251000+0000", 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653327011 2024-09-14T07:01:14.645 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251118+0000", 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000118446 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172726 3.8 3:143e5002:::rbd_data.114fd4a72a77.0000000000000194:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703378552, 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3998113490000001, 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.646 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172726 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.647 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851474+0000", 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2139999999999999e-06 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.648 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851477+0000", 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.208e-06 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851480+0000", 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4700000000000001e-06 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852248+0000", 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00076849899999999996 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.649 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852257+0000", 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6840000000000002e-06 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597699+0000", 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74544160400000004 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597706+0000", 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4170000000000003e-06 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.650 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251171+0000", 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65346452 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251284+0000", 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113498 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.651 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172760 3.8 3:16edf585:::rbd_data.114fd4a72a77.00000000000001b6:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703355525999999, 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3999111539999998, 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172760 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.652 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:14.653 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851497+0000", 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1209999999999999e-06 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851501+0000", 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3279999999999999e-06 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.654 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851505+0000", 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.275e-06 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852297+0000", 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079227500000000005 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852306+0000", 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6659999999999997e-06 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.655 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597736+0000", 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74543018500000002 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597743+0000", 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4290000000000004e-06 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251309+0000", 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6535660569999999 2024-09-14T07:01:14.656 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251407+0000", 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7603999999999996e-05 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172775 3.8 3:104e7293:::rbd_data.114fd4a72a77.00000000000001c5:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703327455, 2024-09-14T07:01:14.657 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4000188580000001, 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172775 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.658 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6100000000000002e-07 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851531+0000", 2024-09-14T07:01:14.659 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2760000000000003e-06 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851536+0000", 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2640000000000002e-06 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851538+0000", 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5289999999999998e-06 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.660 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852358+0000", 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081989599999999997 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852368+0000", 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597767+0000", 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74539930200000004 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.661 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597771+0000", 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4039999999999997e-06 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251441+0000", 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6536695479999999 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251543+0000", 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000101919 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.662 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172778 3.8 3:12efabcb:::rbd_data.114fd4a72a77.00000000000001c8:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703309344000001, 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400145024, 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.663 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172778 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.664 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3900000000000001e-07 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851543+0000", 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851546+0000", 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.665 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851549+0000", 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4530000000000001e-06 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852399+0000", 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00085052200000000004 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852407+0000", 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.238e-06 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.666 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597814+0000", 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74540710700000001 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597821+0000", 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251576+0000", 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653755171 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.667 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251687+0000", 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011061 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172814 3.8 3:166e6639:::rbd_data.114fd4a72a77.00000000000001ec:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 24.703299339000001, 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400279984, 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:14.668 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172814 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.669 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592075+0000", 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74052290899999995 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592081+0000", 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6339999999999998e-06 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:14.670 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592090+0000", 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5929999999999995e-06 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592097+0000", 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4930000000000003e-06 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597854+0000", 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0057574890000000002 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.671 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597858+0000", 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.228e-06 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251725+0000", 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6538667419999999 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251832+0000", 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000106896 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.672 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.673 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:14.673 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:14.673 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:14.674 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:14.673+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:14.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:01:14.775 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:14.774+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:14.783 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:14.783 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:14.783 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:14.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:01:14.876 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:14.876+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:14.939 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:14.939 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:14.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:01:14.991 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:14.991+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:15.057 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.651713266000002, 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:01:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.651651426000001, 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:01:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:01:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.651618249999999, 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:01:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.651495378999996, 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:01:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:01:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:01:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.651460630000003, 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:01:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:01:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:01:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:01:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.651436701999998, 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:01:15.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:01:15.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:01:15.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.651414467999999, 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.651381499999999, 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:01:15.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:01:15.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.651030728999999, 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.096 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:01:15.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:01:15.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:15.100 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.650926545999994, 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:01:15.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:01:15.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.650886036000003, 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.109 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.650860915999999, 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:01:15.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:01:15.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.650706619000005, 2024-09-14T07:01:15.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:01:15.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:01:15.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.650673623000003, 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.650650717999994, 2024-09-14T07:01:15.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.650600714999996, 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:01:15.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:01:15.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:01:15.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.455868662999997, 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.133 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:01:15.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:15.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:15.137 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.451162148999998, 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:15.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:01:15.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.447778483999997, 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:15.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:01:15.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:01:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:01:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:01:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.146 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.447758749000002, 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:15.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:01:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:01:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:15.151 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:15.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:01:15.153 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:15.152+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:15.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:01:15.253 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:15.252+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:15.303 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:15.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:01:15.319 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:15.340 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T07:01:15.348 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:15.353 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:15.353+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:15.373 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:15.394 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:15.394 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:15.414 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:15.437 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:15.437 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:15.449 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:15.449 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:15.454 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:15.453+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:15.456 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:15.456 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:15.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:01:15.476 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:15.555 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:15.554+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:15.588 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:15.588 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:15.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:01:15.681 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:15.680+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:15.744 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:15.745 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:15.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:01:15.757 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:15.756+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:15.857 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:15.857+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:15.901 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:15.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:01:15.959 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:15.958+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:16.057 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:16.058 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:16.060 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:16.059+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:16.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:01:16.161 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:16.160+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:16.174 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:16.174 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:16.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:01:16.261 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:16.261+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:16.290 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:16.290 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:16.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:01:16.363 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:16.362+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:16.463 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:16.463+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:16.464 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:16.464 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:16.473 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:01:16.564 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:16.564+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:16.584 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:16.584 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:16.593 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:01:16.677 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:16.676+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:16.738 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:16.739 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:16.747 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:01:16.766 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:16.765+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:16.866 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:16.866+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:16.894 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:16.894 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:16.903 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:01:16.967 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:16.966+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:17.060 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:17.061 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:17.068 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:17.067+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:17.071 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:01:17.169 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:17.169+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:17.216 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:17.216 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:17.225 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:01:17.310 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:17.309+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:17.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T07:01:17.381 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:17.381 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:17.391 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:01:17.412 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:17.411+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:17.476 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:17.475+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:17.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:01:17.538 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:17.538 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:17.547 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:01:17.573 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:17.572+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:17.621 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:17.638 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:17.638 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:17.654 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:17.670 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:17.674 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:17.673+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:17.693 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:17.693 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:17.702 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:01:17.705 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:17.705 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:17.722 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:17.723 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:17.739 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:17.740 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:17.792 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:17.791+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4272.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 117.04432998, 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033611000000000001, 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.810 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764148+0000", 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.811 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764149+0000", 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8299999999999997e-07 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764156+0000", 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4059999999999999e-06 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764186+0000", 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0110000000000001e-05 2024-09-14T07:01:17.812 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764206+0000", 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.969e-05 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764279+0000", 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2390000000000003e-05 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.764307+0000", 2024-09-14T07:01:17.813 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8436999999999999e-05 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767455+0000", 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031481859999999999 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:20.767510+0000", 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4398000000000003e-05 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.814 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1843 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 98.971210713999994, 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033481800000000001, 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1843 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.815 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837268+0000", 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.816 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837271+0000", 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4869999999999998e-06 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837286+0000", 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5043e-05 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837332+0000", 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6041999999999997e-05 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.817 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837458+0000", 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012578200000000001 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837606+0000", 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014753300000000001 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837698+0000", 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2283999999999999e-05 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.818 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 3", 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837839+0000", 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00014071099999999999 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838565+0000", 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072644199999999999 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840552+0000", 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0019868160000000002 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.819 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840583+0000", 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0691999999999997e-05 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840616+0000", 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3348e-05 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.820 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1846 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 10485760~10485760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 98.970954061, 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003360207, 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1846 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.821 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837524+0000", 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837526+0000", 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.809e-06 2024-09-14T07:01:17.822 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837535+0000", 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5629999999999992e-06 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837576+0000", 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1471999999999999e-05 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.837582+0000", 2024-09-14T07:01:17.823 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.178e-06 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838260+0000", 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00067794999999999997 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.838339+0000", 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8213999999999993e-05 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.824 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.840885+0000", 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0025460209999999999 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1875 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,zero 15728640~901152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 98.944666987000005, 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003145728, 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:17.825 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1875 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863811+0000", 2024-09-14T07:01:17.826 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863814+0000", 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5160000000000001e-06 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863826+0000", 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1673e-05 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863868+0000", 2024-09-14T07:01:17.827 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1839999999999999e-05 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863889+0000", 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1662e-05 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864039+0000", 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015017399999999999 2024-09-14T07:01:17.828 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864138+0000", 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8610999999999995e-05 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.866957+0000", 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002819252 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.829 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:1878 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,zero 14680064~1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 98.944542373000004, 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031018450000000002, 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 1878 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.830 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863936+0000", 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.831 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863937+0000", 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.336e-06 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863946+0000", 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8300000000000002e-06 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863968+0000", 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1416999999999998e-05 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.832 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.863984+0000", 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6143000000000001e-05 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864471+0000", 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00048670700000000001 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.864555+0000", 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.4034000000000002e-05 2024-09-14T07:01:17.833 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:38.867038+0000", 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0024833780000000001 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:2052 3.b e31/24 3:d4a83724:::rbd_data.114fe955cd3f.0000000000000007:head v 31'32, mlcod=30'29)", 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 98.449923210999998, 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0044040329999999999, 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.834 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358555+0000", 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.835 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358556+0000", 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4400000000000001e-07 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358561+0000", 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.417e-06 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358568+0000", 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8680000000000001e-06 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.836 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358576+0000", 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2649999999999996e-06 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358633+0000", 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6416000000000003e-05 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.358651+0000", 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8572000000000001e-05 2024-09-14T07:01:17.837 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362917+0000", 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0042663579999999996 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:39.362959+0000", 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1792999999999998e-05 2024-09-14T07:01:17.838 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:3391 3.b e32/24 3:d199f48e:::rbd_data.114fe2819363.000000000000004d:head v 32'39, mlcod=32'36)", 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 97.175520551000005, 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033607699999999999, 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.839 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632958+0000", 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632959+0000", 2024-09-14T07:01:17.840 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7499999999999999e-07 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632964+0000", 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3879999999999999e-06 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.632994+0000", 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9541999999999999e-05 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.841 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633002+0000", 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.541e-06 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633086+0000", 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3813999999999994e-05 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.633112+0000", 2024-09-14T07:01:17.842 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5737000000000001e-05 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636281+0000", 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031692859999999999 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:40.636319+0000", 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7687e-05 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:7688 3.15 e35/24 3:ac07335a:::rbd_data.114f54fbf0ed.0000000000000069:head v 35'12, mlcod=33'9)", 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 94.180904940999994, 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033116980000000001, 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627574+0000", 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.92e-07 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627581+0000", 2024-09-14T07:01:17.845 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9489999999999999e-06 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627594+0000", 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3078000000000001e-05 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627600+0000", 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9880000000000001e-06 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627648+0000", 2024-09-14T07:01:17.846 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8424000000000003e-05 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.627668+0000", 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9667e-05 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630845+0000", 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031770119999999999 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.630885+0000", 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0388e-05 2024-09-14T07:01:17.847 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10218 3.5 e35/24 3:a44ca967:::rbd_data.114f54fbf0ed.000000000000003b:head v 35'58, mlcod=35'56)", 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.990732475000002, 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003163117, 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.848 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817746+0000", 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817747+0000", 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1900000000000002e-07 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.849 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817944+0000", 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00019724799999999999 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817957+0000", 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2918999999999999e-05 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.850 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.817971+0000", 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4688000000000001e-05 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818017+0000", 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5880000000000001e-05 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.818037+0000", 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9905e-05 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.851 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820878+0000", 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0028409490000000002 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.820909+0000", 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0908999999999998e-05 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.852 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:10639 3.b e35/24 3:d1495d3a:::rbd_data.114f54fbf0ed.0000000000000054:head v 35'59, mlcod=35'57)", 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.941599185000001, 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030433399999999998, 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.853 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866879+0000", 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866880+0000", 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4099999999999998e-07 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866885+0000", 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6480000000000002e-06 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.854 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866894+0000", 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7510000000000003e-06 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866900+0000", 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.998e-06 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866956+0000", 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5980000000000003e-05 2024-09-14T07:01:17.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.866975+0000", 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.8740999999999999e-05 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869892+0000", 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029173860000000001 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:43.869923+0000", 2024-09-14T07:01:17.856 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0195000000000001e-05 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:17445 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'63, mlcod=35'61)", 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.573380220999994, 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0048148360000000003, 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.857 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235098+0000", 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.235100+0000", 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.469e-06 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.858 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236962+0000", 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001861995 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236977+0000", 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5152e-05 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.236983+0000", 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2770000000000002e-06 2024-09-14T07:01:17.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237060+0000", 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7132999999999995e-05 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.237083+0000", 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3235e-05 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239864+0000", 2024-09-14T07:01:17.860 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0027802690000000001 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.239913+0000", 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9305999999999999e-05 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:19249 3.7 e35/24 3:e3e4c8d1:::rbd_data.114f54fbf0ed.000000000000000b:head v 35'42, mlcod=33'40)", 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.452811281999999, 2024-09-14T07:01:17.861 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0033226890000000002, 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355667+0000", 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.862 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.355669+0000", 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.333e-06 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356663+0000", 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00099482600000000009 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356684+0000", 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0633e-05 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.863 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356693+0000", 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4499999999999993e-06 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356764+0000", 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.0525999999999995e-05 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.356789+0000", 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5075999999999998e-05 2024-09-14T07:01:17.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358955+0000", 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021658570000000002 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.358990+0000", 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4987999999999998e-05 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.865 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21055 3.12 e35/31 3:4c812946:::rbd_data.114f54fbf0ed.0000000000000017:head v 35'115, mlcod=35'113)", 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.321895401000006, 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0040730109999999996, 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.866 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486583+0000", 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.486584+0000", 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2359999999999999e-06 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488227+0000", 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001642653 2024-09-14T07:01:17.867 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488247+0000", 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9692999999999999e-05 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488256+0000", 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3889999999999994e-06 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488335+0000", 2024-09-14T07:01:17.868 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8799000000000002e-05 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.488362+0000", 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.741e-05 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490608+0000", 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0022456580000000002 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.869 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.490656+0000", 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8173000000000001e-05 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21059 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'18, mlcod=35'16)", 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.292858139000003, 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.003993758, 2024-09-14T07:01:17.870 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515620+0000", 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.515621+0000", 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.8400000000000002e-07 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517561+0000", 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001939323 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.872 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517582+0000", 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.12e-05 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517592+0000", 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0128e-05 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517648+0000", 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5594999999999999e-05 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.873 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.517664+0000", 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6592999999999999e-05 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519559+0000", 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0018947230000000001 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.519614+0000", 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5211999999999999e-05 2024-09-14T07:01:17.874 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21063 3.5 e35/24 3:a600ce1b:::rbd_data.114f54fbf0ed.000000000000001b:head v 35'64, mlcod=35'62)", 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.267088716999993, 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031920550000000001, 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.875 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541390+0000", 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.541391+0000", 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3e-06 2024-09-14T07:01:17.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542944+0000", 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001553282 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542960+0000", 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5264000000000002e-05 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.877 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.542966+0000", 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4289999999999998e-06 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543011+0000", 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5336000000000001e-05 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.543028+0000", 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6782e-05 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.878 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544532+0000", 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001503372 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.544582+0000", 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0290000000000001e-05 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21111 3.5 e35/24 3:a29d6e50:::rbd_data.114f54fbf0ed.0000000000000003:head v 35'68, mlcod=35'66)", 2024-09-14T07:01:17.879 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.233730574000006, 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030600570000000001, 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.880 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574748+0000", 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574749+0000", 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574755+0000", 2024-09-14T07:01:17.881 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0939999999999996e-06 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574767+0000", 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2051e-05 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574774+0000", 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.72e-06 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.882 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574828+0000", 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3968999999999998e-05 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.574849+0000", 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0783000000000001e-05 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577777+0000", 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002928557 2024-09-14T07:01:17.883 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.577808+0000", 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0836000000000001e-05 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21323 3.15 e35/24 3:af8ad19a:::rbd_data.114f54fbf0ed.0000000000000019:head v 35'26, mlcod=35'24)", 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.069522793000004, 2024-09-14T07:01:17.884 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0034272030000000002, 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738956+0000", 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.885 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738957+0000", 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9899999999999999e-07 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738977+0000", 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0639e-05 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.738994+0000", 2024-09-14T07:01:17.886 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6651999999999999e-05 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739002+0000", 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.943e-06 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739071+0000", 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8941000000000003e-05 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.887 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.739091+0000", 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9928999999999998e-05 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742354+0000", 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0032638200000000002 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.742383+0000", 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8379999999999999e-05 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.888 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21335 3.5 e35/24 3:a303dbdd:::rbd_data.114f54fbf0ed.0000000000000030:head v 35'72, mlcod=35'70)", 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.058143533000006, 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0031007080000000002, 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.889 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750335+0000", 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750336+0000", 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7700000000000004e-07 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.890 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750343+0000", 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8679999999999999e-06 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750352+0000", 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.5120000000000008e-06 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750358+0000", 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6960000000000002e-06 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.891 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750409+0000", 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0980000000000003e-05 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750426+0000", 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7672e-05 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753392+0000", 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002965718 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.892 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753436+0000", 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3684999999999999e-05 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21342 3.7 e35/24 3:e24bb0d6:::rbd_data.114f54fbf0ed.0000000000000050:head v 35'49, mlcod=35'47)", 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.058095839000003, 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0030781649999999999, 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.893 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.894 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750383+0000", 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750389+0000", 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9050000000000002e-06 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750398+0000", 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3960000000000006e-06 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.895 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750403+0000", 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3839999999999996e-06 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750453+0000", 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9625e-05 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750473+0000", 2024-09-14T07:01:17.896 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0652000000000001e-05 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753415+0000", 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0029417250000000001 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.753461+0000", 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5828999999999998e-05 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.897 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_repop(client.4431.0:21339 3.15 e35/24 3:ae2bcbe4:::rbd_data.114f54fbf0ed.000000000000004d:head v 35'28, mlcod=35'25)", 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 93.057753513999998, 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0054741920000000001, 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.898 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750725+0000", 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3099999999999998e-07 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750730+0000", 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9690000000000002e-06 2024-09-14T07:01:17.899 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750739+0000", 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7730000000000005e-06 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750745+0000", 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8100000000000003e-06 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750811+0000", 2024-09-14T07:01:17.900 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5775e-05 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.750821+0000", 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.059e-05 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756175+0000", 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0053533599999999997 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:17.901 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.756199+0000", 2024-09-14T07:01:17.902 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4383999999999999e-05 2024-09-14T07:01:17.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.902 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.902 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:17.902 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:17.902 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:17.904 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:01:17.904 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:17.903+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:18.004 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:18.004+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:18.048 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:18.049 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:18.049 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:18.049 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:18.057 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:01:18.140 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:18.140+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:18.201 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:18.201 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:18.201 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:18.201 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:18.201 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:18.207 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:18.206+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:18.209 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:01:18.307 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:18.307+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:18.355 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-14T07:01:18.357 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:18.357 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:18.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:18.357 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:18.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.357 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172493 3.8 3:11834377:::rbd_data.114fd4a72a77.0000000000000129:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.357 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503788592999999, 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.395516846, 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172493 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.358 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.359 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851241+0000", 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164e-06 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851244+0000", 2024-09-14T07:01:18.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1259999999999998e-06 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851247+0000", 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851660+0000", 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041335800000000001 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851667+0000", 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9809999999999997e-06 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597160+0000", 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549261700000002 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597167+0000", 2024-09-14T07:01:18.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2629999999999997e-06 2024-09-14T07:01:18.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246685+0000", 2024-09-14T07:01:18.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649518327 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246756+0000", 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1210000000000004e-05 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.364 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172499 3.8 3:1336cd84:::rbd_data.114fd4a72a77.000000000000012f:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503774285999999, 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3956566389999998, 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172499 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.365 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:18.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.22e-07 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851255+0000", 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.114e-06 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851259+0000", 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2909999999999999e-06 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851261+0000", 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.655e-06 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851691+0000", 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043023900000000001 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851695+0000", 2024-09-14T07:01:18.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4000000000000001e-06 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597190+0000", 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745495146 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597194+0000", 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7050000000000001e-06 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246801+0000", 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649607445 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246911+0000", 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000109422 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172507 3.8 3:17c8295f:::rbd_data.114fd4a72a77.0000000000000136:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503763734, 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3957446349999998, 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172507 2024-09-14T07:01:18.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851265+0000", 2024-09-14T07:01:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851266+0000", 2024-09-14T07:01:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:01:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851269+0000", 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0079999999999998e-06 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851271+0000", 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.531e-06 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851725+0000", 2024-09-14T07:01:18.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00045350300000000001 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851732+0000", 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9600000000000003e-06 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597223+0000", 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745490926 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597230+0000", 2024-09-14T07:01:18.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4499999999999998e-06 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246943+0000", 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649713363 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247009+0000", 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5656999999999998e-05 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172606 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 14336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503753526000001, 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.39623221, 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172606 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:18.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851276+0000", 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9399999999999993e-07 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851279+0000", 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0070000000000002e-06 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851282+0000", 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9210000000000001e-06 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851756+0000", 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000474005 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851759+0000", 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3670000000000001e-06 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597256+0000", 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549659599999996 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597260+0000", 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247047+0000", 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649786975 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247140+0000", 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3449999999999995e-05 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247507+0000", 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000366664 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172607 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 40960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503736239999999, 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3965954819999999, 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172607 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7000000000000001e-07 2024-09-14T07:01:18.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851294+0000", 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3710000000000001e-06 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851297+0000", 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7850000000000002e-06 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851309+0000", 2024-09-14T07:01:18.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1148000000000001e-05 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851788+0000", 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00047951899999999997 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851803+0000", 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5074e-05 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597290+0000", 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548693700000002 2024-09-14T07:01:18.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597298+0000", 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3590000000000008e-06 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247532+0000", 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.650233759 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247590+0000", 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8091000000000003e-05 2024-09-14T07:01:18.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247887+0000", 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029716900000000001 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172608 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 67584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503707895000002, 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3970747800000001, 2024-09-14T07:01:18.388 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172608 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851321+0000", 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0600000000000001e-07 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851322+0000", 2024-09-14T07:01:18.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.751e-06 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851331+0000", 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1799999999999996e-06 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851334+0000", 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3409999999999999e-06 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851830+0000", 2024-09-14T07:01:18.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00049571000000000001 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851833+0000", 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6069999999999999e-06 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597326+0000", 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549257700000005 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597333+0000", 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247917+0000", 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6505841430000001 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248020+0000", 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102581 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248395+0000", 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037526200000000001 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172609 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 94208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503688249, 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3975014290000001, 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.394 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172609 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9200000000000002e-07 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851342+0000", 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.451e-06 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851346+0000", 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9999999999999998e-06 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851348+0000", 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5780000000000001e-06 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851868+0000", 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051942399999999999 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.397 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851876+0000", 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6850000000000007e-06 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597371+0000", 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745494502 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597378+0000", 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6129999999999998e-06 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248428+0000", 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.651049502 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248482+0000", 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3767999999999997e-05 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248841+0000", 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00035961400000000002 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.399 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172610 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 120832~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503667445000001, 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3980250019999998, 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172610 2024-09-14T07:01:18.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6800000000000002e-07 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851362+0000", 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4300000000000001e-06 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851366+0000", 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.811e-06 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.402 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851369+0000", 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4739999999999999e-06 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851901+0000", 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053255399999999997 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851905+0000", 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4170000000000001e-06 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.403 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597404+0000", 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549902099999998 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597408+0000", 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.527e-06 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248892+0000", 2024-09-14T07:01:18.404 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65148359 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249005+0000", 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113397 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249386+0000", 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038051300000000002 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.405 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172611 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 147456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503649166999999, 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3985748930000002, 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172611 2024-09-14T07:01:18.406 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.407 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.79e-07 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851381+0000", 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7379999999999999e-06 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851385+0000", 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5750000000000001e-06 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.408 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851387+0000", 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851934+0000", 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00054657299999999996 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.409 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851942+0000", 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2549999999999997e-06 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597437+0000", 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549549800000003 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597445+0000", 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5839999999999997e-06 2024-09-14T07:01:18.410 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249454+0000", 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6520086940000001 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249576+0000", 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012215899999999999 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249954+0000", 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037802300000000001 2024-09-14T07:01:18.411 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172612 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 174080~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503637569999999, 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3988136949999999, 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.412 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172612 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.413 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851392+0000", 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851395+0000", 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2449999999999998e-06 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.414 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851398+0000", 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4660000000000002e-06 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851978+0000", 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058031599999999999 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851984+0000", 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1319999999999999e-06 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.415 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597469+0000", 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548469799999995 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597473+0000", 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8510000000000001e-06 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249990+0000", 2024-09-14T07:01:18.416 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6525175509999999 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250091+0000", 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010111900000000001 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250204+0000", 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113034 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172613 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 200704~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503627597000001, 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3989707839999999, 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172613 2024-09-14T07:01:18.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.05e-07 2024-09-14T07:01:18.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851402+0000", 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.026e-06 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851405+0000", 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2440000000000002e-06 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851407+0000", 2024-09-14T07:01:18.420 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3920000000000001e-06 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852016+0000", 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060825699999999996 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852024+0000", 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0639999999999994e-06 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.421 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597505+0000", 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548078699999998 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597513+0000", 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1659999999999994e-06 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250226+0000", 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652713648 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.422 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250279+0000", 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2880999999999997e-05 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250371+0000", 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2114000000000006e-05 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172614 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 227328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:18.423 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503617787, 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3991723779999998, 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172614 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.424 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851411+0000", 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.425 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851412+0000", 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4900000000000004e-07 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851415+0000", 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.942e-06 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.426 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851417+0000", 2024-09-14T07:01:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000002e-06 2024-09-14T07:01:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852054+0000", 2024-09-14T07:01:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00063717699999999995 2024-09-14T07:01:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.427 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852058+0000", 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6739999999999999e-06 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597543+0000", 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548541400000001 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597548+0000", 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6140000000000002e-06 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.428 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250407+0000", 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652859388 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250487+0000", 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9523000000000004e-05 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250583+0000", 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6026999999999994e-05 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.429 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172615 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 253952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503608128, 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3993917979999999, 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172615 2024-09-14T07:01:18.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:18.431 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851421+0000", 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669999999999999e-06 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851425+0000", 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0110000000000002e-06 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851428+0000", 2024-09-14T07:01:18.432 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.452e-06 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852092+0000", 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000664497 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852099+0000", 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1199999999999996e-06 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597585+0000", 2024-09-14T07:01:18.433 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548563599999995 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597593+0000", 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7989999999999994e-06 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250631+0000", 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653038477 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250689+0000", 2024-09-14T07:01:18.434 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7192999999999999e-05 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250812+0000", 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000123344 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172719 3.8 3:10ac98f7:::rbd_data.114fd4a72a77.000000000000018d:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503597312, 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3995141609999999, 2024-09-14T07:01:18.435 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172719 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.436 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:01:18.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851432+0000", 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851435+0000", 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1980000000000001e-06 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851442+0000", 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9419999999999998e-06 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.438 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852133+0000", 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069105100000000004 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852139+0000", 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597622+0000", 2024-09-14T07:01:18.439 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548300599999995 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597635+0000", 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3183000000000001e-05 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250847+0000", 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6532118730000001 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.440 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250945+0000", 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7770999999999996e-05 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172725 3.8 3:16c5e555:::rbd_data.114fd4a72a77.0000000000000193:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503570672999999, 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3996608569999998, 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.441 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172725 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.442 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.443 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851459+0000", 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.626e-06 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851466+0000", 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8349999999999998e-06 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851469+0000", 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.444 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852192+0000", 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072263799999999995 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852201+0000", 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2129999999999997e-06 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597665+0000", 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74546438299999995 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.445 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597673+0000", 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6410000000000002e-06 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251000+0000", 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653327011 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251118+0000", 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000118446 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.446 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172726 3.8 3:143e5002:::rbd_data.114fd4a72a77.0000000000000194:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503555418000001, 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3998113490000001, 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.447 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172726 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.448 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851474+0000", 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2139999999999999e-06 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.449 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851477+0000", 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.208e-06 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851480+0000", 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4700000000000001e-06 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852248+0000", 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00076849899999999996 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.450 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852257+0000", 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6840000000000002e-06 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597699+0000", 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74544160400000004 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597706+0000", 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4170000000000003e-06 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.451 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251171+0000", 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65346452 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251284+0000", 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113498 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172760 3.8 3:16edf585:::rbd_data.114fd4a72a77.00000000000001b6:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:18.452 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503532392, 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3999111539999998, 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172760 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.453 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851497+0000", 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1209999999999999e-06 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.454 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851501+0000", 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3279999999999999e-06 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851505+0000", 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.275e-06 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852297+0000", 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079227500000000005 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.455 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852306+0000", 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6659999999999997e-06 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597736+0000", 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74543018500000002 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597743+0000", 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4290000000000004e-06 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.456 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251309+0000", 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6535660569999999 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251407+0000", 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7603999999999996e-05 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.457 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172775 3.8 3:104e7293:::rbd_data.114fd4a72a77.00000000000001c5:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503504321000001, 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4000188580000001, 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172775 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.458 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.459 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6100000000000002e-07 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851531+0000", 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2760000000000003e-06 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851536+0000", 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2640000000000002e-06 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.460 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851538+0000", 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5289999999999998e-06 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852358+0000", 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081989599999999997 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852368+0000", 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:01:18.461 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597767+0000", 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74539930200000004 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597771+0000", 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4039999999999997e-06 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251441+0000", 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6536695479999999 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.462 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251543+0000", 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000101919 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172778 3.8 3:12efabcb:::rbd_data.114fd4a72a77.00000000000001c8:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503486209999998, 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400145024, 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.463 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172778 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.464 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3900000000000001e-07 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851543+0000", 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851546+0000", 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:01:18.465 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851549+0000", 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4530000000000001e-06 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852399+0000", 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00085052200000000004 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852407+0000", 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.238e-06 2024-09-14T07:01:18.466 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597814+0000", 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74540710700000001 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597821+0000", 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.467 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251576+0000", 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653755171 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251687+0000", 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011061 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.468 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172814 3.8 3:166e6639:::rbd_data.114fd4a72a77.00000000000001ec:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 28.503476204999998, 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400279984, 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172814 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:18.469 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592075+0000", 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74052290899999995 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:18.470 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592081+0000", 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6339999999999998e-06 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592090+0000", 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5929999999999995e-06 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592097+0000", 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4930000000000003e-06 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.471 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597854+0000", 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0057574890000000002 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597858+0000", 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.228e-06 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251725+0000", 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6538667419999999 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:18.472 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251832+0000", 2024-09-14T07:01:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000106896 2024-09-14T07:01:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:18.473 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:18.473 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:18.473 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:01:18.474 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:01:18.474 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T07:01:18.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T07:01:18.476 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:18.473+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:18.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:01:18.575 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:18.574+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:18.625 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:18.625 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:18.625 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:18.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:01:18.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T07:01:18.698 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:18.697+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:18.777 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:18.776+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:18.804 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:18.804 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:18.804 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:18.804 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:18.804 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:18.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:01:18.836 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T07:01:18.879 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:18.878+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:18.979 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:18.978+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:18.987 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:18.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.581323201000004, 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:18.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:18.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:01:18.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:01:18.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:01:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:18.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.581261361000003, 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:18.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:01:18.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:18.999 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.581228185000001, 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:01:19.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:01:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.581105313999998, 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:01:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:01:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:01:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:01:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.581070565000005, 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:01:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:01:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:01:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:01:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.581046637, 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:01:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:01:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.581024403000001, 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:01:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:01:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:01:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.580991435000001, 2024-09-14T07:01:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:01:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:01:19.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:01:19.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.580640664000001, 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:01:19.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.040 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.580536480999996, 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:01:19.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:19.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:01:19.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:01:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:01:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:01:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:01:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:01:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:01:19.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:01:19.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:01:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:01:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.580495971000005, 2024-09-14T07:01:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:01:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:01:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:01:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:01:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:01:19.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:01:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:01:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:01:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:01:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:01:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:01:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:01:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:01:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:01:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:01:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:01:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.580470851000001, 2024-09-14T07:01:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:01:19.059 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:01:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.060 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:01:19.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.580316554000007, 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:01:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:19.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:01:19.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.580283558000005, 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:01:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:01:19.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:01:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:01:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:01:19.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:01:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:01:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.078 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.580260652999996, 2024-09-14T07:01:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:01:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:01:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:19.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:01:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:01:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.086 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.580210649999998, 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:01:19.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:01:19.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.385478598000006, 2024-09-14T07:01:19.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:01:19.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.380772084, 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:01:19.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.377388418999999, 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:01:19.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:01:19.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.377368684000004, 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:19.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:01:19.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.113 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:19.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:01:19.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T07:01:19.117 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:19.114+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:19.215 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:19.215+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:19.267 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:19.267 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:19.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:01:19.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T07:01:19.354 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:19.353+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:19.406 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:19.406 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:19.406 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:19.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:01:19.418 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:19.417+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:19.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T07:01:19.519 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:19.518+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:19.530 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:01:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.795078054, 2024-09-14T07:01:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:01:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:01:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:01:19.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:19.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:01:19.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:176998 3.1c 3:3b6a06c0:::rbd_data.114f2b0c87e5.00000000000000af:21 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:19.538 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.637320814999999, 2024-09-14T07:01:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030101810999999999, 2024-09-14T07:01:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176998 2024-09-14T07:01:19.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1099999999999995e-07 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891479+0000", 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-14T07:01:19.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891486+0000", 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7449999999999995e-06 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891493+0000", 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4829999999999996e-06 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905071+0000", 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013577878999999999 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905181+0000", 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110005 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.921576+0000", 2024-09-14T07:01:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016394855 2024-09-14T07:01:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177029 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636779170000001, 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027087837, 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177029 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:19.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892022+0000", 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7780000000000002e-06 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892026+0000", 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-14T07:01:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892035+0000", 2024-09-14T07:01:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7169999999999997e-06 2024-09-14T07:01:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892041+0000", 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.918935+0000", 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026893675999999998 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919012+0000", 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7599e-05 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919103+0000", 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0734000000000002e-05 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177030 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 40960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.550 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636735863999998, 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027219149000000002, 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177030 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3300000000000002e-07 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892062+0000", 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0259999999999999e-06 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892069+0000", 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1409999999999999e-06 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892075+0000", 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5330000000000002e-06 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919128+0000", 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027053549999999999 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919196+0000", 2024-09-14T07:01:19.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7316000000000004e-05 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919278+0000", 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2050000000000002e-05 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177031 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636704492, 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027358328000000001, 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.556 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177031 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892091+0000", 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9900000000000001e-07 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892093+0000", 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7180000000000001e-06 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892100+0000", 2024-09-14T07:01:19.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7630000000000003e-06 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892106+0000", 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2589999999999997e-06 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919302+0000", 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027195568 2024-09-14T07:01:19.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919368+0000", 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6030000000000003e-05 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919448+0000", 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0490999999999999e-05 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177032 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636673209000001, 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027491581000000001, 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177032 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892122+0000", 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892124+0000", 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3010000000000001e-06 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892131+0000", 2024-09-14T07:01:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8600000000000004e-06 2024-09-14T07:01:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892137+0000", 2024-09-14T07:01:19.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5690000000000004e-06 2024-09-14T07:01:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919472+0000", 2024-09-14T07:01:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027335818000000001 2024-09-14T07:01:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919539+0000", 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6610000000000001e-05 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919613+0000", 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3930000000000005e-05 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177033 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636648619999999, 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027624723, 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177033 2024-09-14T07:01:19.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892148+0000", 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0889999999999998e-06 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892157+0000", 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9719999999999998e-06 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892163+0000", 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1240000000000004e-06 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919636+0000", 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027473329000000001 2024-09-14T07:01:19.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919698+0000", 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2385999999999995e-05 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919771+0000", 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2384000000000004e-05 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177034 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 147456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636623343, 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027770280000000001, 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177034 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892172+0000", 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892174+0000", 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6620000000000001e-06 2024-09-14T07:01:19.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892180+0000", 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0079999999999999e-06 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892185+0000", 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0490000000000003e-06 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919795+0000", 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027610151999999999 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919864+0000", 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.834e-05 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919941+0000", 2024-09-14T07:01:19.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7627999999999996e-05 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177035 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 174080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636592514, 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027885375, 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177035 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892205+0000", 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5239999999999999e-06 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892211+0000", 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749999999999999e-06 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892217+0000", 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3399999999999997e-06 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919964+0000", 2024-09-14T07:01:19.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027746815000000001 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920025+0000", 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1424e-05 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920087+0000", 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2438000000000001e-05 2024-09-14T07:01:19.581 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177036 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 200704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636570224, 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028015455000000002, 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.582 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177036 2024-09-14T07:01:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892225+0000", 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:01:19.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892227+0000", 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1579999999999999e-06 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892241+0000", 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4353e-05 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892248+0000", 2024-09-14T07:01:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4230000000000002e-06 2024-09-14T07:01:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920111+0000", 2024-09-14T07:01:19.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027863849 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920176+0000", 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4873e-05 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920240+0000", 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3377e-05 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.587 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177037 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 227328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636536792000001, 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028139249000000002, 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.588 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177037 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892261+0000", 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4889999999999998e-06 2024-09-14T07:01:19.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892268+0000", 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7939999999999997e-06 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892290+0000", 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2444999999999999e-05 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920265+0000", 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027974987 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920332+0000", 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6935999999999997e-05 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920397+0000", 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5081999999999998e-05 2024-09-14T07:01:19.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177038 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.636494617, 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028249414, 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:19.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177038 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:19.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2900000000000004e-07 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892303+0000", 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6790000000000001e-06 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892312+0000", 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2839999999999995e-06 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892318+0000", 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1460000000000004e-06 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920421+0000", 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028103197999999999 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920486+0000", 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4960000000000001e-05 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920549+0000", 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3618e-05 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177325 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.627990750999999, 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026889606999999999, 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177325 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900808+0000", 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900816+0000", 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4409999999999993e-06 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900820+0000", 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927681+0000", 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026860346 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927693+0000", 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2685e-05 2024-09-14T07:01:19.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177326 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.627969214, 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026903018000000001, 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177326 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900826+0000", 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900827+0000", 2024-09-14T07:01:19.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.544e-06 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900845+0000", 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7484999999999999e-05 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900849+0000", 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5630000000000002e-06 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927720+0000", 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02687121 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927728+0000", 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8800000000000008e-06 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177327 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.627938864000001, 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026904081999999999, 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177327 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900858+0000", 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.962e-06 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900869+0000", 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.148e-05 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900874+0000", 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2250000000000002e-06 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927752+0000", 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026878655000000001 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927760+0000", 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3719999999999999e-06 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.612 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177365 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.627583940000001, 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026950818000000001, 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177365 2024-09-14T07:01:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:19.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0100000000000001e-07 2024-09-14T07:01:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901212+0000", 2024-09-14T07:01:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3799999999999999e-06 2024-09-14T07:01:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901221+0000", 2024-09-14T07:01:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5809999999999997e-06 2024-09-14T07:01:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901225+0000", 2024-09-14T07:01:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928151+0000", 2024-09-14T07:01:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026925722999999999 2024-09-14T07:01:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928161+0000", 2024-09-14T07:01:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0596e-05 2024-09-14T07:01:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177366 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.627564782, 2024-09-14T07:01:19.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026960812000000001, 2024-09-14T07:01:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177366 2024-09-14T07:01:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.91e-07 2024-09-14T07:01:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901231+0000", 2024-09-14T07:01:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.409e-06 2024-09-14T07:01:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901240+0000", 2024-09-14T07:01:19.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5290000000000008e-06 2024-09-14T07:01:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901244+0000", 2024-09-14T07:01:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1250000000000003e-06 2024-09-14T07:01:19.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928182+0000", 2024-09-14T07:01:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026937492 2024-09-14T07:01:19.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928191+0000", 2024-09-14T07:01:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:01:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177367 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.632 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.627545521999998, 2024-09-14T07:01:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026971434999999998, 2024-09-14T07:01:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177367 2024-09-14T07:01:19.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:01:19.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901251+0000", 2024-09-14T07:01:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6750000000000001e-06 2024-09-14T07:01:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901259+0000", 2024-09-14T07:01:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:01:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901263+0000", 2024-09-14T07:01:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1119999999999998e-06 2024-09-14T07:01:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928213+0000", 2024-09-14T07:01:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026949533000000001 2024-09-14T07:01:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928220+0000", 2024-09-14T07:01:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7800000000000001e-06 2024-09-14T07:01:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.642 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177404 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.627161386000001, 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026884965, 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177404 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.643 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901635+0000", 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5710000000000001e-06 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901644+0000", 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6969999999999999e-06 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901648+0000", 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0690000000000003e-06 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928511+0000", 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026863267999999999 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928518+0000", 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0600000000000002e-06 2024-09-14T07:01:19.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177405 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.627141896000001, 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026894992, 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:19.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177405 2024-09-14T07:01:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:19.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901654+0000", 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901663+0000", 2024-09-14T07:01:19.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0280000000000006e-06 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901668+0000", 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1160000000000001e-06 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928541+0000", 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026873107 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:19.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928548+0000", 2024-09-14T07:01:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.973e-06 2024-09-14T07:01:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:19.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:19.653 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:19.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:01:19.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T07:01:19.656 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:19.654+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:19.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:01:19.794 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:19.794+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:19.805 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:19.806 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:19.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:01:19.881 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:19.881 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:19.882 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:19.880+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:19.897 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:19.898 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:19.914 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:19.914 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:19.931 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:19.931 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:19.948 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:19.948 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:19.957 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:19.956+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:19.965 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:19.966 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:19.982 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:19.983 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:19.988 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:19.988 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:19.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:01:19.999 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:20.058 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:20.057+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:20.107 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:20.107 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:20.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:01:20.159 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:20.158+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:20.259 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:20.258+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:20.263 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:20.263 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:20.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:01:20.360 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:20.360+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:20.437 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:20.438 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:20.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:01:20.461 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:20.460+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:20.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:01:20.559 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:20.559 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:20.562 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:20.561+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:20.569 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:01:20.663 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:20.662+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:20.673 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:20.673 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:20.682 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:01:20.765 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:20.765+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:20.826 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:20.826 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:20.834 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:01:20.876 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:20.875+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:20.964 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:20.964+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:20.978 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:20.978 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:20.987 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:01:21.065 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:21.065+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:21.131 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:21.131 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:21.140 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:01:21.184 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:21.183+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:21.243 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:21.243 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:21.252 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:01:21.293 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:21.292+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:21.366 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T07:01:21.368 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:21.366+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:21.403 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:21.404 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:21.413 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:01:21.497 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:21.496+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:21.558 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:21.559 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:21.567 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:01:21.568 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:21.568+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:21.669 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:21.669+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:21.711 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:21.711 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:21.720 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:01:21.804 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:21.804+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:21.865 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:21.865 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:21.872 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:21.871+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:21.874 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:01:21.973 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:21.972+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:22.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:01:22.017 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:22.017 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:22.026 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:01:22.073 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:22.072+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:22.130 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:22.130 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:22.139 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:01:22.149 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:22.167 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:22.167 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:22.180 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:22.179+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:22.184 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:22.184 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:22.200 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:22.200 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:22.218 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:22.218 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:22.236 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:22.236 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:22.253 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:22.253 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:22.270 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:22.270 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:22.274 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:22.273+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:22.288 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:22.288 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:22.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:22.288 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:22.288 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.288 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172493 3.8 3:11834377:::rbd_data.114fd4a72a77.0000000000000129:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.288 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:22.288 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.43499207, 2024-09-14T07:01:22.288 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.395516846, 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172493 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.289 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851240+0000", 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.290 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851241+0000", 2024-09-14T07:01:22.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.164e-06 2024-09-14T07:01:22.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.291 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.291 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.291 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851244+0000", 2024-09-14T07:01:22.291 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1259999999999998e-06 2024-09-14T07:01:22.291 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851247+0000", 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5959999999999998e-06 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851660+0000", 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00041335800000000001 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.292 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851667+0000", 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9809999999999997e-06 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597160+0000", 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549261700000002 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597167+0000", 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2629999999999997e-06 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.293 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246685+0000", 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649518327 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246756+0000", 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1210000000000004e-05 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.294 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172499 3.8 3:1336cd84:::rbd_data.114fd4a72a77.000000000000012f:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434977762999999, 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3956566389999998, 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172499 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.295 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.296 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851254+0000", 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.22e-07 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851255+0000", 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.114e-06 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851259+0000", 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2909999999999999e-06 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.297 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851261+0000", 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.655e-06 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851691+0000", 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00043023900000000001 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851695+0000", 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4000000000000001e-06 2024-09-14T07:01:22.298 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597190+0000", 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745495146 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597194+0000", 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7050000000000001e-06 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246801+0000", 2024-09-14T07:01:22.299 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649607445 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246911+0000", 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000109422 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172507 3.8 3:17c8295f:::rbd_data.114fd4a72a77.0000000000000136:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:22.300 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434967211, 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3957446349999998, 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172507 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.301 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851264+0000", 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851265+0000", 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.302 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851266+0000", 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0300000000000001e-06 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851269+0000", 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0079999999999998e-06 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851271+0000", 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.531e-06 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.303 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851725+0000", 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00045350300000000001 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851732+0000", 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9600000000000003e-06 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597223+0000", 2024-09-14T07:01:22.304 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745490926 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597230+0000", 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4499999999999998e-06 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.246943+0000", 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649713363 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.305 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247009+0000", 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5656999999999998e-05 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172606 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 14336~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434957003000001, 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.39623221, 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.306 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172606 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.307 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851275+0000", 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851276+0000", 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.9399999999999993e-07 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.308 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851279+0000", 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0070000000000002e-06 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851282+0000", 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9210000000000001e-06 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851756+0000", 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000474005 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.309 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851759+0000", 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3670000000000001e-06 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597256+0000", 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549659599999996 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597260+0000", 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:01:22.310 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247047+0000", 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.649786975 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247140+0000", 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.3449999999999995e-05 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247507+0000", 2024-09-14T07:01:22.311 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000366664 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172607 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 40960~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434939716999999, 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3965954819999999, 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.312 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172607 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.313 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851292+0000", 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7000000000000001e-07 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851294+0000", 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3710000000000001e-06 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.314 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851297+0000", 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7850000000000002e-06 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851309+0000", 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1148000000000001e-05 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851788+0000", 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00047951899999999997 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.315 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851803+0000", 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5074e-05 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597290+0000", 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548693700000002 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597298+0000", 2024-09-14T07:01:22.316 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3590000000000008e-06 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247532+0000", 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.650233759 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247590+0000", 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8091000000000003e-05 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.317 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247887+0000", 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00029716900000000001 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172608 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 67584~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434911372000002, 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3970747800000001, 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.318 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172608 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851320+0000", 2024-09-14T07:01:22.319 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851321+0000", 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0600000000000001e-07 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851322+0000", 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.751e-06 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.320 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851331+0000", 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1799999999999996e-06 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851334+0000", 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3409999999999999e-06 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851830+0000", 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00049571000000000001 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.321 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851833+0000", 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6069999999999999e-06 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597326+0000", 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549257700000005 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597333+0000", 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.3220000000000004e-06 2024-09-14T07:01:22.322 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.247917+0000", 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6505841430000001 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248020+0000", 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000102581 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248395+0000", 2024-09-14T07:01:22.323 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037526200000000001 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172609 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 94208~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434891725999996, 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3975014290000001, 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.324 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172609 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.325 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851340+0000", 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9200000000000002e-07 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851342+0000", 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.451e-06 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851346+0000", 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9999999999999998e-06 2024-09-14T07:01:22.326 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851348+0000", 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5780000000000001e-06 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851868+0000", 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00051942399999999999 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851876+0000", 2024-09-14T07:01:22.327 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6850000000000007e-06 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597371+0000", 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.745494502 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597378+0000", 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6129999999999998e-06 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.328 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248428+0000", 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.651049502 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248482+0000", 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3767999999999997e-05 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248841+0000", 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00035961400000000002 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.329 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172610 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 120832~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434870922000002, 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3980250019999998, 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172610 2024-09-14T07:01:22.330 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.331 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851361+0000", 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6800000000000002e-07 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851362+0000", 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4300000000000001e-06 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851366+0000", 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.811e-06 2024-09-14T07:01:22.332 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851369+0000", 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4739999999999999e-06 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851901+0000", 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00053255399999999997 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.333 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851905+0000", 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4170000000000001e-06 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597404+0000", 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549902099999998 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597408+0000", 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.527e-06 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.334 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.248892+0000", 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65148359 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249005+0000", 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113397 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249386+0000", 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00038051300000000002 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.335 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172611 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 147456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434852644000003, 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3985748930000002, 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172611 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.336 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851379+0000", 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.79e-07 2024-09-14T07:01:22.337 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851381+0000", 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7379999999999999e-06 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851385+0000", 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5750000000000001e-06 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.338 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851387+0000", 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5150000000000001e-06 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851934+0000", 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00054657299999999996 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851942+0000", 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2549999999999997e-06 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.339 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597437+0000", 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74549549800000003 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597445+0000", 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5839999999999997e-06 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249454+0000", 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6520086940000001 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.340 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249576+0000", 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00012215899999999999 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249954+0000", 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00037802300000000001 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172612 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 174080~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434841046999999, 2024-09-14T07:01:22.341 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3988136949999999, 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172612 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.342 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851391+0000", 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851392+0000", 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.079e-06 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.343 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851395+0000", 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2449999999999998e-06 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851398+0000", 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4660000000000002e-06 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851978+0000", 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00058031599999999999 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.344 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851984+0000", 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1319999999999999e-06 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597469+0000", 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548469799999995 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597473+0000", 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8510000000000001e-06 2024-09-14T07:01:22.345 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.249990+0000", 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6525175509999999 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250091+0000", 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010111900000000001 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250204+0000", 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113034 2024-09-14T07:01:22.346 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172613 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 200704~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434831074000002, 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3989707839999999, 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.347 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172613 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.348 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851401+0000", 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.05e-07 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851402+0000", 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.026e-06 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851405+0000", 2024-09-14T07:01:22.349 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2440000000000002e-06 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851407+0000", 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3920000000000001e-06 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852016+0000", 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00060825699999999996 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.350 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852024+0000", 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0639999999999994e-06 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597505+0000", 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548078699999998 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597513+0000", 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1659999999999994e-06 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.351 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250226+0000", 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652713648 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250279+0000", 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2880999999999997e-05 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250371+0000", 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2114000000000006e-05 2024-09-14T07:01:22.352 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172614 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 227328~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434821264, 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3991723779999998, 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.353 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172614 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851410+0000", 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.354 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851411+0000", 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1e-07 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851412+0000", 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4900000000000004e-07 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851415+0000", 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.942e-06 2024-09-14T07:01:22.355 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851417+0000", 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4600000000000002e-06 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852054+0000", 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00063717699999999995 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852058+0000", 2024-09-14T07:01:22.356 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6739999999999999e-06 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597543+0000", 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548541400000001 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597548+0000", 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6140000000000002e-06 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.357 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250407+0000", 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.652859388 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250487+0000", 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.9523000000000004e-05 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250583+0000", 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.6026999999999994e-05 2024-09-14T07:01:22.358 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172615 3.8 3:15a32c80:::rbd_data.114f2b0c87e5.0000000000000004:21 [read 253952~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434811605, 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3993917979999999, 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.359 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172615 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.360 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851420+0000", 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0200000000000001e-07 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851421+0000", 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0669999999999999e-06 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851425+0000", 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0110000000000002e-06 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.361 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851428+0000", 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.452e-06 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852092+0000", 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000664497 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852099+0000", 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1199999999999996e-06 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.362 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597585+0000", 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548563599999995 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597593+0000", 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.7989999999999994e-06 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250631+0000", 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653038477 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.363 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250689+0000", 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7192999999999999e-05 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250812+0000", 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000123344 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172719 3.8 3:10ac98f7:::rbd_data.114fd4a72a77.000000000000018d:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:22.364 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434800789000001, 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3995141609999999, 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172719 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:22.365 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851431+0000", 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.11e-07 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.366 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851432+0000", 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851435+0000", 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1980000000000001e-06 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851442+0000", 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9419999999999998e-06 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.367 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852133+0000", 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069105100000000004 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852139+0000", 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8459999999999996e-06 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597622+0000", 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74548300599999995 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.368 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597635+0000", 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.3183000000000001e-05 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250847+0000", 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6532118730000001 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.250945+0000", 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7770999999999996e-05 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.369 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172725 3.8 3:16c5e555:::rbd_data.114fd4a72a77.0000000000000193:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434774150000003, 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3996608569999998, 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.370 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172725 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.371 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851458+0000", 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9400000000000001e-07 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851459+0000", 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.626e-06 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851466+0000", 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8349999999999998e-06 2024-09-14T07:01:22.372 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851469+0000", 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852192+0000", 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00072263799999999995 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852201+0000", 2024-09-14T07:01:22.373 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.2129999999999997e-06 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597665+0000", 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74546438299999995 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597673+0000", 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.6410000000000002e-06 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.374 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251000+0000", 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653327011 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251118+0000", 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000118446 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.375 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172726 3.8 3:143e5002:::rbd_data.114fd4a72a77.0000000000000194:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434758895000002, 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3998113490000001, 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172726 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.376 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851473+0000", 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:01:22.377 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851474+0000", 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2139999999999999e-06 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851477+0000", 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.208e-06 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851480+0000", 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4700000000000001e-06 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.378 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852248+0000", 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00076849899999999996 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852257+0000", 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6840000000000002e-06 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597699+0000", 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74544160400000004 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.379 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597706+0000", 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4170000000000003e-06 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251171+0000", 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.65346452 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251284+0000", 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000113498 2024-09-14T07:01:22.380 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172760 3.8 3:16edf585:::rbd_data.114fd4a72a77.00000000000001b6:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434735869000001, 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3999111539999998, 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.381 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172760 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.382 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851496+0000", 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851497+0000", 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1209999999999999e-06 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851501+0000", 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3279999999999999e-06 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.383 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851505+0000", 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.275e-06 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852297+0000", 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00079227500000000005 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852306+0000", 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.6659999999999997e-06 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.384 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597736+0000", 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74543018500000002 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597743+0000", 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.4290000000000004e-06 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251309+0000", 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6535660569999999 2024-09-14T07:01:22.385 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251407+0000", 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.7603999999999996e-05 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172775 3.8 3:104e7293:::rbd_data.114fd4a72a77.00000000000001c5:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434707797999998, 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4000188580000001, 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.386 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172775 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.387 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851524+0000", 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6100000000000002e-07 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851531+0000", 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.2760000000000003e-06 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.388 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851536+0000", 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2640000000000002e-06 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851538+0000", 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5289999999999998e-06 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852358+0000", 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00081989599999999997 2024-09-14T07:01:22.389 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852368+0000", 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.4590000000000004e-06 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597767+0000", 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74539930200000004 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597771+0000", 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4039999999999997e-06 2024-09-14T07:01:22.390 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251441+0000", 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6536695479999999 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251543+0000", 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000101919 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.391 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172778 3.8 3:12efabcb:::rbd_data.114fd4a72a77.00000000000001c8:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434689687000002, 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400145024, 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172778 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.392 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851542+0000", 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.3900000000000001e-07 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.393 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851543+0000", 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0869999999999999e-06 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851546+0000", 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0630000000000002e-06 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851549+0000", 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4530000000000001e-06 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.394 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852399+0000", 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00085052200000000004 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.852407+0000", 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.238e-06 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597814+0000", 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74540710700000001 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.395 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597821+0000", 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251576+0000", 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.653755171 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251687+0000", 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011061 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.396 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:172814 3.8 3:166e6639:::rbd_data.114fd4a72a77.00000000000001ec:head [sparse-read 0~262144] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 32.434679682000002, 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.400279984, 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:22.397 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 172814 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:49.851552+0000", 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.398 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592075+0000", 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.74052290899999995 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592081+0000", 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6339999999999998e-06 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592090+0000", 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.5929999999999995e-06 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.399 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.592097+0000", 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4930000000000003e-06 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597854+0000", 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0057574890000000002 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:50.597858+0000", 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.228e-06 2024-09-14T07:01:22.400 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251725+0000", 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6538667419999999 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:52.251832+0000", 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000106896 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.401 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:22.402 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:22.402 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:22.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:01:22.404 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:22.402+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:22.504 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:22.503+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:22.516 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:22.517 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:22.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:01:22.605 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:22.604+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:22.632 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:22.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:22.633 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:22.633 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:22.633 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:22.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:01:22.706 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:22.705+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:22.750 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:22.750 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:22.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:22.750 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:22.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.750 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74998 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.750 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:22.750 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.344329501999994, 2024-09-14T07:01:22.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752218700000003, 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74998 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:22.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404078+0000", 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404080+0000", 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404087+0000", 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2679999999999997e-06 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404107+0000", 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9967e-05 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404117+0000", 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7820000000000005e-06 2024-09-14T07:01:22.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404186+0000", 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9626e-05 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5150000000000006e-06 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371587+0000", 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96739218800000004 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371600+0000", 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2938999999999999e-05 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:74999 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.755 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.344267661999993, 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96760228299999995, 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 74999 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404139+0000", 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404140+0000", 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.37e-07 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404144+0000", 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4620000000000001e-06 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404152+0000", 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4500000000000004e-06 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404158+0000", 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404225+0000", 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6618000000000005e-05 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404231+0000", 2024-09-14T07:01:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371732+0000", 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750118699999998 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371742+0000", 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7370000000000001e-06 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75000 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.344234486000005, 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96759637399999998, 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75000 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404173+0000", 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404180+0000", 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.029e-06 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404189+0000", 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7420000000000003e-06 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404195+0000", 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2670000000000003e-06 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404323+0000", 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012830799999999999 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1990000000000003e-06 2024-09-14T07:01:22.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371763+0000", 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743335100000005 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371769+0000", 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75001 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.344111615000003, 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96750860500000002, 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75001 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.766 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404296+0000", 2024-09-14T07:01:22.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404300+0000", 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404312+0000", 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1799e-05 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404319+0000", 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6980000000000001e-06 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404353+0000", 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4499000000000003e-05 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404359+0000", 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9039999999999997e-06 2024-09-14T07:01:22.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371799+0000", 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967439889 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371804+0000", 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3170000000000001e-06 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75002 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.344076865999995, 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96752116300000002, 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75002 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.771 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404330+0000", 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404331+0000", 2024-09-14T07:01:22.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404334+0000", 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404340+0000", 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7469999999999999e-06 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404346+0000", 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0240000000000003e-06 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404381+0000", 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4745e-05 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404386+0000", 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2530000000000004e-06 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371842+0000", 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96745588599999999 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371851+0000", 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.268e-06 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.775 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75003 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.344052938000004, 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751674799999998, 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75003 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404354+0000", 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404355+0000", 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:22.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404357+0000", 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404364+0000", 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9090000000000003e-06 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404369+0000", 2024-09-14T07:01:22.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404411+0000", 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2021999999999998e-05 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404416+0000", 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4060000000000004e-06 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371865+0000", 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967448427 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371871+0000", 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0279999999999997e-06 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75004 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.780 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.344030704000005, 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96751000200000004, 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75004 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404376+0000", 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404377+0000", 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4900000000000001e-07 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404379+0000", 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.108e-06 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404385+0000", 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2990000000000002e-06 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404391+0000", 2024-09-14T07:01:22.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.293e-06 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404439+0000", 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8047999999999998e-05 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404444+0000", 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5260000000000001e-06 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371882+0000", 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96743756000000003 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371886+0000", 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7190000000000001e-06 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75005 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.343997736000006, 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.967491926, 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75005 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404409+0000", 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404410+0000", 2024-09-14T07:01:22.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404413+0000", 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.588e-06 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404420+0000", 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1010000000000003e-06 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404426+0000", 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9519999999999999e-06 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404466+0000", 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0308000000000001e-05 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404471+0000", 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9790000000000001e-06 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371897+0000", 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96742603500000002 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371901+0000", 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4220000000000002e-06 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.790 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75061 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.343646965000005, 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96717191499999999, 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75061 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404760+0000", 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404761+0000", 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.082e-06 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404766+0000", 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404781+0000", 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4557e-05 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404789+0000", 2024-09-14T07:01:22.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8089999999999993e-06 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404846+0000", 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7705999999999999e-05 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404853+0000", 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8920000000000003e-06 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371927+0000", 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96707398300000003 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371932+0000", 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.729e-06 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75062 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.795 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.343542782, 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96708343299999999, 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75062 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404864+0000", 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404865+0000", 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2099999999999996e-07 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404868+0000", 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3139999999999999e-06 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404877+0000", 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.276e-06 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404883+0000", 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.708e-06 2024-09-14T07:01:22.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7212000000000001e-05 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404937+0000", 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6869999999999997e-06 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371943+0000", 2024-09-14T07:01:22.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96700568600000003 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371948+0000", 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75063 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.343502271999995, 2024-09-14T07:01:22.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706896399999998, 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75063 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404905+0000", 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404908+0000", 2024-09-14T07:01:22.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8490000000000002e-06 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404915+0000", 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404921+0000", 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7899999999999996e-06 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404965+0000", 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3380000000000001e-05 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404970+0000", 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7089999999999999e-06 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371967+0000", 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96699633200000001 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371974+0000", 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2439999999999996e-06 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75064 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.343477152000006, 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96706015899999997, 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75064 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404930+0000", 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404933+0000", 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1619999999999998e-06 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404939+0000", 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.404944+0000", 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230000000000001e-06 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405186+0000", 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000241998 2024-09-14T07:01:22.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405193+0000", 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7599999999999997e-06 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371986+0000", 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96679309300000005 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.371990+0000", 2024-09-14T07:01:22.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75065 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.343322854999997, 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96693684300000005, 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75065 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405084+0000", 2024-09-14T07:01:22.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405085+0000", 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405089+0000", 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405100+0000", 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1771e-05 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405107+0000", 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7279999999999998e-06 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405231+0000", 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012423800000000001 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405237+0000", 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372016+0000", 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96677859799999999 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372021+0000", 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2979999999999999e-06 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75066 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.343289858999995, 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966934607, 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75066 2024-09-14T07:01:22.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405117+0000", 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405118+0000", 2024-09-14T07:01:22.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405120+0000", 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6960000000000001e-06 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405127+0000", 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405133+0000", 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-06 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405262+0000", 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012956400000000001 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405267+0000", 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6029999999999998e-06 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372047+0000", 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966779944 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372052+0000", 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9869999999999999e-06 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.819 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75067 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.343266954000001, 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96694144500000001, 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75067 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405140+0000", 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405141+0000", 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405143+0000", 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.244e-06 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405149+0000", 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5910000000000001e-06 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405154+0000", 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0350000000000001e-06 2024-09-14T07:01:22.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405289+0000", 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405295+0000", 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1909999999999999e-06 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372077+0000", 2024-09-14T07:01:22.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.966782103 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372082+0000", 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8659999999999997e-06 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:75068 3.1f 3.24dff31f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:22.824 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.343216951000002, 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96691300700000005, 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 75068 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:22.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405190+0000", 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405191+0000", 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405194+0000", 2024-09-14T07:01:22.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9289999999999998e-06 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405200+0000", 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.832e-06 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405206+0000", 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7610000000000004e-06 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405337+0000", 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013069899999999999 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:57.405344+0000", 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8090000000000004e-06 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372099+0000", 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.96675516699999997 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:58.372103+0000", 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2980000000000002e-06 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:114834 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 2295808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 71.148484898999996, 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063882385999999999, 2024-09-14T07:01:22.829 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 114834 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599922+0000", 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599923+0000", 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599925+0000", 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-14T07:01:22.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599931+0000", 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.599936+0000", 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662835+0000", 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.062898447999999996 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.662936+0000", 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010102099999999999 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.663805+0000", 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086875099999999996 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115041 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1584128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 71.143778385000005, 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063787669000000005, 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115041 2024-09-14T07:01:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.604629+0000", 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606399+0000", 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001770308 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606404+0000", 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5560000000000001e-06 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606413+0000", 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4630000000000007e-06 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.606420+0000", 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668275+0000", 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.061855332999999998 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668341+0000", 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5828999999999996e-05 2024-09-14T07:01:22.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.668416+0000", 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5454e-05 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115053 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1756160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 71.140394720000003, 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063915369999999999, 2024-09-14T07:01:22.838 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115053 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608012+0000", 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608013+0000", 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608015+0000", 2024-09-14T07:01:22.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.996e-06 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608021+0000", 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8429999999999998e-06 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608026+0000", 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671557+0000", 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.06353135 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671620+0000", 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3342000000000004e-05 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.671928+0000", 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030757799999999997 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:115054 3.3 3:c0de4375:::rbd_data.114f480080ee.0000000000000000:14 [read 1770496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e59)", 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 71.140374984999994, 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064213653999999995, 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:22.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 115054 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.608032+0000", 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613309+0000", 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052772380000000001 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613314+0000", 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999997e-06 2024-09-14T07:01:22.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613323+0000", 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9789999999999999e-06 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.613330+0000", 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0960000000000003e-06 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672010+0000", 2024-09-14T07:01:22.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.058679374999999999 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672092+0000", 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1959000000000006e-05 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:11.672246+0000", 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154287 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:22.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:22.848 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:22.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:01:22.849 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:22.848+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:22.949 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:22.949+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:22.996 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:22.997 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:22.997 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:22.997 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:23.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:01:23.050 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:23.050+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:23.151 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:23.150+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:23.152 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:23.152 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:23.152 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:23.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:01:23.252 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:23.251+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:23.270 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:23.270 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:23.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:23.270 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:23.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.270 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:01:23.270 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.53494447, 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:01:23.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:01:23.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:23.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:176998 3.1c 3:3b6a06c0:::rbd_data.114f2b0c87e5.00000000000000af:21 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:23.277 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.377187231000001, 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030101810999999999, 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176998 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:23.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1099999999999995e-07 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891479+0000", 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891486+0000", 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7449999999999995e-06 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891493+0000", 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4829999999999996e-06 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905071+0000", 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013577878999999999 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905181+0000", 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110005 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.921576+0000", 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016394855 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177029 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.376645586000002, 2024-09-14T07:01:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027087837, 2024-09-14T07:01:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177029 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892022+0000", 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7780000000000002e-06 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892026+0000", 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-14T07:01:23.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892035+0000", 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7169999999999997e-06 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892041+0000", 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.918935+0000", 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026893675999999998 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919012+0000", 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7599e-05 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919103+0000", 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0734000000000002e-05 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177030 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 40960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.37660228, 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027219149000000002, 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177030 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3300000000000002e-07 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892062+0000", 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0259999999999999e-06 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892069+0000", 2024-09-14T07:01:23.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1409999999999999e-06 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892075+0000", 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5330000000000002e-06 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919128+0000", 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027053549999999999 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919196+0000", 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7316000000000004e-05 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919278+0000", 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2050000000000002e-05 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177031 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.292 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.376570907999998, 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027358328000000001, 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177031 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892091+0000", 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9900000000000001e-07 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892093+0000", 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7180000000000001e-06 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892100+0000", 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7630000000000003e-06 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892106+0000", 2024-09-14T07:01:23.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2589999999999997e-06 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919302+0000", 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027195568 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919368+0000", 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6030000000000003e-05 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919448+0000", 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0490999999999999e-05 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177032 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.376539624999999, 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027491581000000001, 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177032 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892122+0000", 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892124+0000", 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3010000000000001e-06 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892131+0000", 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8600000000000004e-06 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892137+0000", 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5690000000000004e-06 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919472+0000", 2024-09-14T07:01:23.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027335818000000001 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919539+0000", 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6610000000000001e-05 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919613+0000", 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3930000000000005e-05 2024-09-14T07:01:23.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177033 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.376515036000001, 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027624723, 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177033 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892148+0000", 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0889999999999998e-06 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892157+0000", 2024-09-14T07:01:23.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9719999999999998e-06 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892163+0000", 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1240000000000004e-06 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919636+0000", 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027473329000000001 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919698+0000", 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2385999999999995e-05 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919771+0000", 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2384000000000004e-05 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177034 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 147456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.306 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.376489759000002, 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027770280000000001, 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177034 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892172+0000", 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892174+0000", 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6620000000000001e-06 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892180+0000", 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0079999999999999e-06 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892185+0000", 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0490000000000003e-06 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919795+0000", 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027610151999999999 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919864+0000", 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.834e-05 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919941+0000", 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7627999999999996e-05 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177035 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 174080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.376458929999998, 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027885375, 2024-09-14T07:01:23.311 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177035 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892205+0000", 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5239999999999999e-06 2024-09-14T07:01:23.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892211+0000", 2024-09-14T07:01:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749999999999999e-06 2024-09-14T07:01:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892217+0000", 2024-09-14T07:01:23.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3399999999999997e-06 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919964+0000", 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027746815000000001 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920025+0000", 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1424e-05 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920087+0000", 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2438000000000001e-05 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177036 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 200704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.376436640000001, 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028015455000000002, 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177036 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892225+0000", 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892227+0000", 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1579999999999999e-06 2024-09-14T07:01:23.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892241+0000", 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4353e-05 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892248+0000", 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4230000000000002e-06 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920111+0000", 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027863849 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920176+0000", 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4873e-05 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920240+0000", 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3377e-05 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177037 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 227328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.376403207999999, 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028139249000000002, 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177037 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892261+0000", 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4889999999999998e-06 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892268+0000", 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7939999999999997e-06 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892290+0000", 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2444999999999999e-05 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920265+0000", 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027974987 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920332+0000", 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6935999999999997e-05 2024-09-14T07:01:23.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920397+0000", 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5081999999999998e-05 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177038 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.376361033000002, 2024-09-14T07:01:23.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028249414, 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177038 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2900000000000004e-07 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892303+0000", 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6790000000000001e-06 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892312+0000", 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2839999999999995e-06 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892318+0000", 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1460000000000004e-06 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920421+0000", 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028103197999999999 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920486+0000", 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4960000000000001e-05 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920549+0000", 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3618e-05 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177325 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.367857166999997, 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026889606999999999, 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177325 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900808+0000", 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900816+0000", 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4409999999999993e-06 2024-09-14T07:01:23.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900820+0000", 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927681+0000", 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026860346 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927693+0000", 2024-09-14T07:01:23.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2685e-05 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177326 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.367835630000002, 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026903018000000001, 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177326 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:23.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900826+0000", 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900827+0000", 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.544e-06 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900845+0000", 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7484999999999999e-05 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900849+0000", 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5630000000000002e-06 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927720+0000", 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02687121 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927728+0000", 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8800000000000008e-06 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177327 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.367805279999999, 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026904081999999999, 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177327 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900858+0000", 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.962e-06 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900869+0000", 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.148e-05 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900874+0000", 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2250000000000002e-06 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927752+0000", 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026878655000000001 2024-09-14T07:01:23.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927760+0000", 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3719999999999999e-06 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177365 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.367450355999999, 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026950818000000001, 2024-09-14T07:01:23.342 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177365 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0100000000000001e-07 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901212+0000", 2024-09-14T07:01:23.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3799999999999999e-06 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901221+0000", 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5809999999999997e-06 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901225+0000", 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:23.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928151+0000", 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026925722999999999 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928161+0000", 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0596e-05 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177366 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.367431197999998, 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026960812000000001, 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177366 2024-09-14T07:01:23.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.91e-07 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901231+0000", 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.409e-06 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901240+0000", 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5290000000000008e-06 2024-09-14T07:01:23.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901244+0000", 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1250000000000003e-06 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928182+0000", 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026937492 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928191+0000", 2024-09-14T07:01:23.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177367 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.367411937999996, 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026971434999999998, 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177367 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:23.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901251+0000", 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6750000000000001e-06 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901259+0000", 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901263+0000", 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1119999999999998e-06 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928213+0000", 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026949533000000001 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928220+0000", 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7800000000000001e-06 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177404 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.367027802000003, 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026884965, 2024-09-14T07:01:23.355 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177404 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901635+0000", 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5710000000000001e-06 2024-09-14T07:01:23.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901644+0000", 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6969999999999999e-06 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901648+0000", 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0690000000000003e-06 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928511+0000", 2024-09-14T07:01:23.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026863267999999999 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928518+0000", 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0600000000000002e-06 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177405 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:23.359 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.367008312000003, 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026894992, 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177405 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901654+0000", 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901663+0000", 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0280000000000006e-06 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901668+0000", 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1160000000000001e-06 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928541+0000", 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026873107 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928548+0000", 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.973e-06 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.363 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:23.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:01:23.365 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:23.364+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:23.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:01:23.465 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:23.465+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:23.534 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:23.534 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:23.534 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:23.534 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:23.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:01:23.587 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:23.587+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:23.651 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:23.652 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:23.652 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:23.652 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:23.652 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:23.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:01:23.666 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:23.666+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:23.767 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:23.766+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:23.808 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:23.808 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:23.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:23.808 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119746 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1298432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.261994555000001, 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067798875999999994, 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119746 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545019+0000", 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545020+0000", 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.782e-06 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545026+0000", 2024-09-14T07:01:23.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9390000000000002e-06 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545031+0000", 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5770000000000004e-06 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612601+0000", 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067570682000000007 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612683+0000", 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1378999999999995e-05 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612817+0000", 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013415000000000001 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119747 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.261976136000001, 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068036012000000007, 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119747 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546792+0000", 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017554960000000001 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546797+0000", 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1950000000000002e-06 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546808+0000", 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0675e-05 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546815+0000", 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612854+0000", 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066039158000000001 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612930+0000", 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.613073+0000", 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000142798 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119754 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1413120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.259961435999998, 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067735654000000006, 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119754 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549973+0000", 2024-09-14T07:01:23.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029215999999999999 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549978+0000", 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3510000000000001e-06 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549988+0000", 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0037000000000001e-05 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549995+0000", 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9709999999999998e-06 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614569+0000", 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064573368000000006 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614650+0000", 2024-09-14T07:01:23.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1909000000000005e-05 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614787+0000", 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013641799999999999 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119764 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1556480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:23.822 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.256789529000002, 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067672107999999995, 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119764 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550224+0000", 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550226+0000", 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1720000000000001e-06 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550233+0000", 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9879999999999998e-06 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550238+0000", 2024-09-14T07:01:23.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1819999999999997e-06 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617715+0000", 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067476628999999996 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617807+0000", 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2899999999999995e-05 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617895+0000", 2024-09-14T07:01:23.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7837999999999995e-05 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119765 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.256767096999994, 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067829500000000001, 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119765 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7e-07 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550248+0000", 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1280000000000002e-06 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550256+0000", 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-14T07:01:23.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550261+0000", 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617926+0000", 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067665280999999994 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617995+0000", 2024-09-14T07:01:23.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9478000000000001e-05 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618075+0000", 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9666000000000006e-05 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119950 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 529408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.256744991000005, 2024-09-14T07:01:23.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067992002999999995, 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119950 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550852+0000", 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058435400000000004 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550857+0000", 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6979999999999997e-06 2024-09-14T07:01:23.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550867+0000", 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0378e-05 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550874+0000", 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8480000000000003e-06 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618106+0000", 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067231714999999997 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618178+0000", 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2572999999999998e-05 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618260+0000", 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1437e-05 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119970 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 816128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.250089834999997, 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067745828999999994, 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119970 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556924+0000", 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0410000000000001e-06 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556927+0000", 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9679999999999998e-06 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556930+0000", 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624487+0000", 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067557392999999993 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624551+0000", 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3861000000000004e-05 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624669+0000", 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117921 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119971 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.250079896000003, 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067869065000000006, 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119971 2024-09-14T07:01:23.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556934+0000", 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.096e-06 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556937+0000", 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.796e-06 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556939+0000", 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624708+0000", 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067768992 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624747+0000", 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8148999999999997e-05 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624802+0000", 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.533e-05 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119972 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 844800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:23.844 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.250070225000002, 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068010762000000002, 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119972 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:23.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556943+0000", 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556944+0000", 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:01:23.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556947+0000", 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9720000000000001e-06 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556949+0000", 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4899999999999999e-06 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624838+0000", 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067888899000000003 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624895+0000", 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7395000000000002e-05 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624953+0000", 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7745999999999999e-05 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119973 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 859136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.250060343000001, 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068210898000000006, 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119973 2024-09-14T07:01:23.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556953+0000", 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556954+0000", 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189999999999999e-06 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556957+0000", 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9490000000000001e-06 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556959+0000", 2024-09-14T07:01:23.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3870000000000002e-06 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624983+0000", 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068023726000000007 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625049+0000", 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6385999999999998e-05 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625163+0000", 2024-09-14T07:01:23.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114221 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119974 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.250050422000001, 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068338594000000003, 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119974 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556963+0000", 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556964+0000", 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556969+0000", 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.04e-06 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556974+0000", 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.639e-06 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625194+0000", 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068220742000000001 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625246+0000", 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1844e-05 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625301+0000", 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4551e-05 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119975 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.250032181999998, 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068554198999999996, 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119975 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556983+0000", 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2560000000000001e-06 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556989+0000", 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2210000000000002e-06 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556994+0000", 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7779999999999998e-06 2024-09-14T07:01:23.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625330+0000", 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068335911999999999 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625396+0000", 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6051999999999997e-05 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625535+0000", 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013860200000000001 2024-09-14T07:01:23.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.862 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119976 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.862 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:23.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.250011186999998, 2024-09-14T07:01:23.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068656164000000006, 2024-09-14T07:01:23.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119976 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557004+0000", 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557010+0000", 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1770000000000003e-06 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557016+0000", 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2650000000000004e-06 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625566+0000", 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068550323999999996 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625604+0000", 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8398000000000001e-05 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625658+0000", 2024-09-14T07:01:23.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3239000000000003e-05 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119977 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.249930767999999, 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068713522999999999, 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119977 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557086+0000", 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3569999999999998e-06 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557094+0000", 2024-09-14T07:01:23.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6829999999999997e-06 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557101+0000", 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625689+0000", 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068587882000000003 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625743+0000", 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4082000000000002e-05 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625795+0000", 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2657000000000001e-05 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119978 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 930816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.249902176000006, 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068826410000000005, 2024-09-14T07:01:23.871 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119978 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557113+0000", 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-14T07:01:23.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557120+0000", 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1079999999999998e-06 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557125+0000", 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625823+0000", 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068698928000000006 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625884+0000", 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0056999999999998e-05 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625937+0000", 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3383e-05 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119979 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 945152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.249881028999994, 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068940788000000003, 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119979 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5199999999999998e-07 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557134+0000", 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.018e-06 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557140+0000", 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557145+0000", 2024-09-14T07:01:23.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7790000000000003e-06 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625966+0000", 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068820972999999994 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626019+0000", 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3828e-05 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626072+0000", 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3078000000000002e-05 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119980 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.249860670999993, 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069053630000000005, 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119980 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557159+0000", 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557167+0000", 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999996e-06 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557172+0000", 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626101+0000", 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068928552000000004 2024-09-14T07:01:23.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626154+0000", 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3338000000000003e-05 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626206+0000", 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1657999999999999e-05 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119981 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.249832021000003, 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069157629999999998, 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119981 2024-09-14T07:01:23.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557183+0000", 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557190+0000", 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2249999999999997e-06 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557195+0000", 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3120000000000001e-06 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626234+0000", 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069038926 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626286+0000", 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2604e-05 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626338+0000", 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1947000000000003e-05 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119982 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 988160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.249810867999997, 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069364004000000007, 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:23.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119982 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:23.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560989+0000", 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037867360000000002 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560994+0000", 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561004+0000", 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3680000000000008e-06 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561011+0000", 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1640000000000004e-06 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626366+0000", 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.065355286999999998 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626433+0000", 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6554e-05 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626566+0000", 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132996 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:191378 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T07:01:23.894 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.138723676000001, 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.95875689900000005, 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 191378 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668290+0000", 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3319999999999999e-06 2024-09-14T07:01:23.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668295+0000", 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7570000000000004e-06 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668306+0000", 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0813e-05 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668311+0000", 2024-09-14T07:01:23.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0839999999999999e-06 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668364+0000", 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2673999999999997e-05 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668371+0000", 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:23.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624394+0000", 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.95602263300000001 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624558+0000", 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000163525 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624767+0000", 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000209555 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.626401+0000", 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016343110000000001 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.626918+0000", 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051661100000000005 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.627005+0000", 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6854999999999994e-05 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:23.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:23.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:23.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.627046+0000", 2024-09-14T07:01:23.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0970999999999998e-05 2024-09-14T07:01:23.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:23.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:23.901 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:23.903 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:01:23.903 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:23.902+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:24.003 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:24.003+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:24.053 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:24.053 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:24.053 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:24.053 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:24.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:01:24.105 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:24.104+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:24.169 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:24.170 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:24.170 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:24.170 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:24.170 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:24.178 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:01:24.222 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:24.221+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:24.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:01:24.287 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:24.287 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.531946871, 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.288 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:01:24.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.933534279, 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:01:24.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:01:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:01:24.300 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.927874636, 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:01:24.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:01:24.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.622038357999998, 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.311 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.622007565999994, 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:01:24.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:01:24.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.621984196, 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.621177686999999, 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.621155294000005, 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:01:24.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.328 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.621135154000001, 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:24.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:01:24.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:01:24.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.620984999000001, 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.620929445000002, 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:01:24.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.620896404000007, 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.345 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.618543318999997, 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.349 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.618520754000002, 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.350 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:24.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:01:24.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.617663758999996, 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:01:24.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:24.358 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.616446737999993, 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:01:24.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:01:24.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.616318898000003, 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:01:24.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.616298805, 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:01:24.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:01:24.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.615271911999997, 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:24.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:24.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138497 3.14 3:298b17d3:::rbd_data.114f74bd3bbf.00000000000002d7:head [set-alloc-hint object_size 262144 write_size 262144,write 218355~43789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.967764629999998, 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97686345699999999, 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138497 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:24.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317983+0000", 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318003+0000", 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0610000000000001e-05 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318022+0000", 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8627e-05 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318030+0000", 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318101+0000", 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1162999999999996e-05 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318108+0000", 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291257+0000", 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97314914100000005 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291396+0000", 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013883799999999999 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291573+0000", 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017716800000000001 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.293147+0000", 2024-09-14T07:01:24.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015740400000000001 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294753+0000", 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00160597 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294792+0000", 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8758999999999999e-05 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:24.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294844+0000", 2024-09-14T07:01:24.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2188000000000002e-05 2024-09-14T07:01:24.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:24.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:24.383 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:24.384 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T07:01:24.385 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:01:24.386 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:24.383+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:24.429 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:24.446 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:24.447 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:24.463 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:24.479 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:24.479 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:24.485 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:24.484+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:24.496 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:24.496 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:24.499 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:24.499 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:24.509 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:01:24.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:24.514 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:24.530 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:24.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:24.548 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:24.601 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:24.600+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:24.653 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:24.654 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:24.662 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:01:24.687 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:24.686+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:24.766 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:24.766 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:24.774 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:01:24.787 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:24.787+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:24.839 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 7, 2] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T07:01:24.839 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T07:01:24.839 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T07:01:24.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T07:01:24.877 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:24.877 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:24.886 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:01:24.888 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:24.888+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:24.989 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:24.988+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:24.990 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:24.990 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:24.999 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:01:25.090 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:25.090+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:25.142 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 128, 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6", 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-14T06:58:59.479055+0000", 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-14T07:01:24.289653+0000", 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-14T06:59:11.718934+0000", 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-14T07:00:26.509032+0000", 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 6, 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 8, 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-14T07:01:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:11.765419+0000", 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:01:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:01:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "18", 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:01:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:01:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:01:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 6.9000000953674316, 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 6.9000000953674316, 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-14T07:01:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.82401084899902344, 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.93999999761581421, 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:18.343475+0000", 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:01:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-14T07:01:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "22", 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:01:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 22, 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:01:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:01:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 3.5, 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 3.5, 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T07:01:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 3.5, 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 3.5, 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.82401084899902344, 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.93999999761581421, 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-14T06:59:27.257805+0000", 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-14T07:01:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-14T07:01:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "128", 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 36, 2024-09-14T07:01:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 128, 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-14T07:01:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-14T07:01:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.309999942779541, 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.309999942779541, 2024-09-14T07:01:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.309999942779541, 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.309999942779541, 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 0.82401084899902344, 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 0.93999999761581421, 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 0.81999999284744263 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c036038d-cffb-4584-8851-69604a31ee40", 2024-09-14T07:01:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0.5920867919921875, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 82, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2081065366 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/2081065366", 2024-09-14T07:01:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/2081065366", 2024-09-14T07:01:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/2081065366", 2024-09-14T07:01:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/2081065366", 2024-09-14T07:01:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "1c80080f-932a-4957-b64b-f30c4bdb9340", 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 99, 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1688050909 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6806/1688050909", 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6810/1688050909", 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/1688050909", 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/1688050909", 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f3544d11-8a0f-498c-a03c-5d34134c9773", 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 99, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6824", 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6825", 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6826", 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6827", 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6830", 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6831", 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6828", 2024-09-14T07:01:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6829", 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1872259061 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6825/1872259061", 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6827/1872259061", 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6831/1872259061", 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6829/1872259061", 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "d2261455-c91b-414b-925f-388f0e094dbe", 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.6944732666015625, 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 99, 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-14T07:01:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1193301030 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/1193301030", 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/1193301030", 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6811/1193301030", 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6807/1193301030", 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4088d23b-0d06-4415-9eba-6fb64355746c", 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 76, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6810", 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6811", 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6812", 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6813", 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6816", 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6817", 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6814", 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6815", 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2283562461 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6811/2283562461", 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6813/2283562461", 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6817/2283562461", 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6815/2283562461", 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "5e27cb53-ecce-4061-bc5a-be599bf18cc4", 2024-09-14T07:01:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 14, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 77, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6818", 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6819", 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6820", 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6821", 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6824", 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6825", 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6822", 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6823", 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2212769626 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6819/2212769626", 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6821/2212769626", 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6825/2212769626", 2024-09-14T07:01:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6823/2212769626", 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ceb9a1f1-f54c-48d5-91d8-2287cac96ceb", 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 0, 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 15, 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 24, 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6826", 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6827", 2024-09-14T07:01:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6828", 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6829", 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6832", 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6833", 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6830", 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6831", 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3197231389 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6827/3197231389", 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6829/3197231389", 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6833/3197231389", 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6831/3197231389", 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "85c59cd6-6e2b-4cc0-852d-d1f075a8826d", 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-14T07:01:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 13, 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 99, 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6800", 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6801", 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6802", 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6803", 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6806", 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6807", 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6804", 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.189:6805", 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2162534312 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.189:6801/2162534312", 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.189:6803/2162534312", 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.189:6807/2162534312", 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.189:6805/2162534312", 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-14T07:01:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.064941+0000", 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.780598+0000", 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-14T07:01:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.127700+0000", 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.730995+0000", 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.864423+0000", 2024-09-14T07:01:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:09.329361+0000", 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 6, 2024-09-14T07:01:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 65536, 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:10.107581+0000", 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 7, 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540719139924082687, 2024-09-14T07:01:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-14T06:59:08.300043+0000", 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.8", 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: 0 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.11", 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.18", 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.b", 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 7, 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 5 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1f", 2024-09-14T07:01:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 5, 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 0 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-14T07:01:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 8, 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 7 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 8, 2024-09-14T07:01:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [ 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 15, 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 3 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:25.223 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:25.225 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:25.225 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:25.226 INFO:tasks.thrashosds.thrasher:j is {'epoch': 128, 'fsid': '32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6', 'created': '2024-09-14T06:58:59.479055+0000', 'modified': '2024-09-14T07:01:24.289653+0000', 'last_up_change': '2024-09-14T06:59:11.718934+0000', 'last_in_change': '2024-09-14T07:00:26.509032+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:59:11.765419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8240108489990234, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:59:18.343475+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.5, 'score_stable': 3.5, 'optimal_score': 1, 'raw_score_acting': 3.5, 'raw_score_stable': 3.5, 'primary_affinity_weighted': 0.8240108489990234, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:59:27.257805+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 36, 'snap_epoch': 128, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 0.8240108489990234, 'average_primary_affinity': 0.9399999976158142, 'average_primary_affinity_weighted': 0.8199999928474426}}], 'osds': [{'osd': 0, 'uuid': 'c036038d-cffb-4584-8851-69604a31ee40', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.5920867919921875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 2081065366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 2081065366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 2081065366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 2081065366}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 2081065366}]}, 'public_addr': '172.21.15.7:6817/2081065366', 'cluster_addr': '172.21.15.7:6819/2081065366', 'heartbeat_back_addr': '172.21.15.7:6823/2081065366', 'heartbeat_front_addr': '172.21.15.7:6821/2081065366', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '1c80080f-932a-4957-b64b-f30c4bdb9340', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6806', 'nonce': 1688050909}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6810', 'nonce': 1688050909}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 1688050909}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 1688050909}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 1688050909}]}, 'public_addr': '172.21.15.7:6806/1688050909', 'cluster_addr': '172.21.15.7:6810/1688050909', 'heartbeat_back_addr': '172.21.15.7:6815/1688050909', 'heartbeat_front_addr': '172.21.15.7:6813/1688050909', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'f3544d11-8a0f-498c-a03c-5d34134c9773', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6824', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6825', 'nonce': 1872259061}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6826', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6827', 'nonce': 1872259061}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6830', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6831', 'nonce': 1872259061}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6828', 'nonce': 1872259061}, {'type': 'v1', 'addr': '172.21.15.7:6829', 'nonce': 1872259061}]}, 'public_addr': '172.21.15.7:6825/1872259061', 'cluster_addr': '172.21.15.7:6827/1872259061', 'heartbeat_back_addr': '172.21.15.7:6831/1872259061', 'heartbeat_front_addr': '172.21.15.7:6829/1872259061', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd2261455-c91b-414b-925f-388f0e094dbe', 'up': 1, 'in': 1, 'weight': 0.6944732666015625, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 1193301030}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 1193301030}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6809', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 1193301030}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6805', 'nonce': 1193301030}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 1193301030}]}, 'public_addr': '172.21.15.7:6801/1193301030', 'cluster_addr': '172.21.15.7:6803/1193301030', 'heartbeat_back_addr': '172.21.15.7:6811/1193301030', 'heartbeat_front_addr': '172.21.15.7:6807/1193301030', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '4088d23b-0d06-4415-9eba-6fb64355746c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6810', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6811', 'nonce': 2283562461}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6812', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6813', 'nonce': 2283562461}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6816', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6817', 'nonce': 2283562461}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6814', 'nonce': 2283562461}, {'type': 'v1', 'addr': '172.21.15.189:6815', 'nonce': 2283562461}]}, 'public_addr': '172.21.15.189:6811/2283562461', 'cluster_addr': '172.21.15.189:6813/2283562461', 'heartbeat_back_addr': '172.21.15.189:6817/2283562461', 'heartbeat_front_addr': '172.21.15.189:6815/2283562461', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '5e27cb53-ecce-4061-bc5a-be599bf18cc4', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6818', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6819', 'nonce': 2212769626}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6820', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6821', 'nonce': 2212769626}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6824', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6825', 'nonce': 2212769626}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6822', 'nonce': 2212769626}, {'type': 'v1', 'addr': '172.21.15.189:6823', 'nonce': 2212769626}]}, 'public_addr': '172.21.15.189:6819/2212769626', 'cluster_addr': '172.21.15.189:6821/2212769626', 'heartbeat_back_addr': '172.21.15.189:6825/2212769626', 'heartbeat_front_addr': '172.21.15.189:6823/2212769626', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceb9a1f1-f54c-48d5-91d8-2287cac96ceb', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6826', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6827', 'nonce': 3197231389}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6828', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6829', 'nonce': 3197231389}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6832', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6833', 'nonce': 3197231389}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6830', 'nonce': 3197231389}, {'type': 'v1', 'addr': '172.21.15.189:6831', 'nonce': 3197231389}]}, 'public_addr': '172.21.15.189:6827/3197231389', 'cluster_addr': '172.21.15.189:6829/3197231389', 'heartbeat_back_addr': '172.21.15.189:6833/3197231389', 'heartbeat_front_addr': '172.21.15.189:6831/3197231389', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '85c59cd6-6e2b-4cc0-852d-d1f075a8826d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6800', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6801', 'nonce': 2162534312}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6802', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6803', 'nonce': 2162534312}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6806', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6807', 'nonce': 2162534312}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6804', 'nonce': 2162534312}, {'type': 'v1', 'addr': '172.21.15.189:6805', 'nonce': 2162534312}]}, 'public_addr': '172.21.15.189:6801/2162534312', 'cluster_addr': '172.21.15.189:6803/2162534312', 'heartbeat_back_addr': '172.21.15.189:6807/2162534312', 'heartbeat_front_addr': '172.21.15.189:6805/2162534312', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.064941+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.780598+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.127700+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.730995+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.864423+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:09.329361+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T06:59:10.107581+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:59:08.300043+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.8', 'osds': [7, 0]}, {'pgid': '3.11', 'osds': [6, 7]}, {'pgid': '3.18', 'osds': [0, 5]}], 'pg_upmap_items': [{'pgid': '3.b', 'mappings': [{'from': 7, 'to': 5}]}, {'pgid': '3.1f', 'mappings': [{'from': 5, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 8, 'length': 7}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 8, 'length': 1}]}], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 15, 'length': 3}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T07:01:25.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:01:25.227 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:01:25.227 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:25.224+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:25.325 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:25.324+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:25.370 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:25.370 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:25.379 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:01:25.425 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:25.425+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:25.498 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:25.498 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:25.507 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:01:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:25.544 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:01:25.547 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:25.547+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:25.555 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-14T07:01:25.381063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":368738291,"num_objects":788,"num_object_clones":40,"num_object_copies":1576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":788,"num_whiteouts":608,"num_read":239836,"num_read_kb":529479,"num_write":22507,"num_write_kb":273226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2099,"num_bytes_recovered":84516687,"num_keys_recovered":235,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19988,"ondisk_log_size":19988,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":384972,"kb_used_data":19496,"kb_used_omap":10,"kb_used_meta":365301,"kb_avail":660217908,"statfs":{"total":676457349120,"available":676063137792,"internally_reserved":0,"allocated":19963904,"data_stored":75102570,"data_compressed":2932887,"data_compressed_allocated":7016448,"data_compressed_original":68728968,"omap_allocated":11157,"internal_metadata":374068331},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-63513461,"num_objects":-178,"num_object_clones":-72,"num_object_copies":-356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-178,"num_whiteouts":-63,"num_read":1198,"num_read_kb":1017,"num_write":192,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019909"},"pg_stats":[{"pgid":"3.1f","version":"124'528","reported_seq":7842,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.369048+0000","last_change":"2024-09-14T07:01:22.369863+0000","last_active":"2024-09-14T07:01:22.369048+0000","last_peered":"2024-09-14T07:01:22.369048+0000","last_clean":"2024-09-14T07:01:22.369048+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:01:22.369048+0000","last_undegraded":"2024-09-14T07:01:22.369048+0000","last_fullsized":"2024-09-14T07:01:22.369048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":528,"log_dups_size":0,"ondisk_log_size":528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:25:58.853527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010297710000000001,"stat_sum":{"num_bytes":30984446,"num_objects":24,"num_object_clones":2,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":8421,"num_read_kb":37326,"num_write":502,"num_write_kb":21659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.1e","version":"121'418","reported_seq":2914,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T07:01:20.359941+0000","last_change":"2024-09-14T07:01:20.360169+0000","last_active":"2024-09-14T07:01:20.359941+0000","last_peered":"2024-09-14T07:01:20.359941+0000","last_clean":"2024-09-14T07:01:20.359941+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:01:20.359941+0000","last_undegraded":"2024-09-14T07:01:20.359941+0000","last_fullsized":"2024-09-14T07:01:20.359941+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:46:14.220018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044560299999999998,"stat_sum":{"num_bytes":1571285,"num_objects":27,"num_object_clones":1,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":23,"num_read":2716,"num_read_kb":8402,"num_write":357,"num_write_kb":7998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"6"}]},{"pgid":"3.1d","version":"126'336","reported_seq":5754,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.377141+0000","last_change":"2024-09-14T07:01:22.377439+0000","last_active":"2024-09-14T07:01:22.377141+0000","last_peered":"2024-09-14T07:01:22.377141+0000","last_clean":"2024-09-14T07:01:22.377141+0000","last_became_active":"2024-09-14T07:00:56.618952+0000","last_became_peered":"2024-09-14T07:00:56.618952+0000","last_unstale":"2024-09-14T07:01:22.377141+0000","last_undegraded":"2024-09-14T07:01:22.377141+0000","last_fullsized":"2024-09-14T07:01:22.377141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:10:49.976562+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0047522780000000004,"stat_sum":{"num_bytes":2498938,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":23,"num_read":5182,"num_read_kb":12861,"num_write":292,"num_write_kb":5166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.1c","version":"126'2031","reported_seq":4724,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.399229+0000","last_change":"2024-09-14T07:01:22.376982+0000","last_active":"2024-09-14T07:01:22.399229+0000","last_peered":"2024-09-14T07:01:22.399229+0000","last_clean":"2024-09-14T07:01:22.399229+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:01:22.399229+0000","last_undegraded":"2024-09-14T07:01:22.399229+0000","last_fullsized":"2024-09-14T07:01:22.399229+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":2031,"log_dups_size":0,"ondisk_log_size":2031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:40:53.011586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042609379999999997,"stat_sum":{"num_bytes":1792776,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":16,"num_read":11631,"num_read_kb":22503,"num_write":3435,"num_write_kb":11219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.1b","version":"125'883","reported_seq":3265,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-14T07:01:21.479402+0000","last_change":"2024-09-14T07:01:21.369665+0000","last_active":"2024-09-14T07:01:21.479402+0000","last_peered":"2024-09-14T07:01:21.479402+0000","last_clean":"2024-09-14T07:01:21.479402+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:01:21.479402+0000","last_undegraded":"2024-09-14T07:01:21.479402+0000","last_fullsized":"2024-09-14T07:01:21.479402+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:47:01.261109+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0072669780000000003,"stat_sum":{"num_bytes":12836912,"num_objects":26,"num_object_clones":1,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":19,"num_read":4411,"num_read_kb":12211,"num_write":1120,"num_write_kb":20062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"7"}]},{"pgid":"3.1a","version":"126'1179","reported_seq":8811,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.457182+0000","last_change":"2024-09-14T07:01:22.376218+0000","last_active":"2024-09-14T07:01:22.457182+0000","last_peered":"2024-09-14T07:01:22.457182+0000","last_clean":"2024-09-14T07:01:22.457182+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:01:22.457182+0000","last_undegraded":"2024-09-14T07:01:22.457182+0000","last_fullsized":"2024-09-14T07:01:22.457182+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1179,"log_dups_size":0,"ondisk_log_size":1179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:19:10.834601+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0035877249999999999,"stat_sum":{"num_bytes":18120475,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":21,"num_read":13011,"num_read_kb":21042,"num_write":1635,"num_write_kb":6895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.19","version":"125'510","reported_seq":4914,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-14T07:01:21.365419+0000","last_change":"2024-09-14T07:01:21.365761+0000","last_active":"2024-09-14T07:01:21.365419+0000","last_peered":"2024-09-14T07:01:21.365419+0000","last_clean":"2024-09-14T07:01:21.365419+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:01:21.365419+0000","last_undegraded":"2024-09-14T07:01:21.365419+0000","last_fullsized":"2024-09-14T07:01:21.365419+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:53:01.958541+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033401590000000001,"stat_sum":{"num_bytes":43733516,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":18,"num_read":4742,"num_read_kb":12493,"num_write":517,"num_write_kb":11830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"7"}]},{"pgid":"3.18","version":"126'656","reported_seq":3712,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.371503+0000","last_change":"2024-09-14T07:01:22.371885+0000","last_active":"2024-09-14T07:01:22.371503+0000","last_peered":"2024-09-14T07:01:22.371503+0000","last_clean":"2024-09-14T07:01:22.371503+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:01:22.371503+0000","last_undegraded":"2024-09-14T07:01:22.371503+0000","last_fullsized":"2024-09-14T07:01:22.371503+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:15.264698+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033626569999999998,"stat_sum":{"num_bytes":3049315,"num_objects":22,"num_object_clones":1,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":17,"num_read":5020,"num_read_kb":10268,"num_write":678,"num_write_kb":6392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.17","version":"124'371","reported_seq":6672,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T07:01:20.471657+0000","last_change":"2024-09-14T07:01:20.360948+0000","last_active":"2024-09-14T07:01:20.471657+0000","last_peered":"2024-09-14T07:01:20.471657+0000","last_clean":"2024-09-14T07:01:20.471657+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:01:20.471657+0000","last_undegraded":"2024-09-14T07:01:20.471657+0000","last_fullsized":"2024-09-14T07:01:20.471657+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:48:43.647882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011962780000000001,"stat_sum":{"num_bytes":402572,"num_objects":29,"num_object_clones":1,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":25,"num_read":7500,"num_read_kb":16678,"num_write":309,"num_write_kb":2845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"6"}]},{"pgid":"3.16","version":"124'409","reported_seq":4863,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.368043+0000","last_change":"2024-09-14T07:01:22.368921+0000","last_active":"2024-09-14T07:01:22.368043+0000","last_peered":"2024-09-14T07:01:22.368043+0000","last_clean":"2024-09-14T07:01:22.368043+0000","last_became_active":"2024-09-14T07:00:36.547481+0000","last_became_peered":"2024-09-14T07:00:36.547481+0000","last_unstale":"2024-09-14T07:01:22.368043+0000","last_undegraded":"2024-09-14T07:01:22.368043+0000","last_fullsized":"2024-09-14T07:01:22.368043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:16:14.542211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030833689999999999,"stat_sum":{"num_bytes":3973636,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":16,"num_read":4902,"num_read_kb":8441,"num_write":351,"num_write_kb":3835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.15","version":"124'342","reported_seq":2454,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.365742+0000","last_change":"2024-09-14T07:01:22.366286+0000","last_active":"2024-09-14T07:01:22.365742+0000","last_peered":"2024-09-14T07:01:22.365742+0000","last_clean":"2024-09-14T07:01:22.365742+0000","last_became_active":"2024-09-14T07:00:56.618968+0000","last_became_peered":"2024-09-14T07:00:56.618968+0000","last_unstale":"2024-09-14T07:01:22.365742+0000","last_undegraded":"2024-09-14T07:01:22.365742+0000","last_fullsized":"2024-09-14T07:01:22.365742+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:08:03.840545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010478460000000001,"stat_sum":{"num_bytes":1783402,"num_objects":24,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":21,"num_read":1776,"num_read_kb":3899,"num_write":303,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.14","version":"125'769","reported_seq":5590,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.367285+0000","last_change":"2024-09-14T07:01:22.367938+0000","last_active":"2024-09-14T07:01:22.367285+0000","last_peered":"2024-09-14T07:01:22.367285+0000","last_clean":"2024-09-14T07:01:22.367285+0000","last_became_active":"2024-09-14T07:00:36.547583+0000","last_became_peered":"2024-09-14T07:00:36.547583+0000","last_unstale":"2024-09-14T07:01:22.367285+0000","last_undegraded":"2024-09-14T07:01:22.367285+0000","last_fullsized":"2024-09-14T07:01:22.367285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:36:24.849780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023907070000000002,"stat_sum":{"num_bytes":14630963,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":21,"num_read":6696,"num_read_kb":12591,"num_write":1040,"num_write_kb":7165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"2.7","version":"0'0","reported_seq":228,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.365052+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:01:22.365052+0000","last_peered":"2024-09-14T07:01:22.365052+0000","last_clean":"2024-09-14T07:01:22.365052+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:01:22.365052+0000","last_undegraded":"2024-09-14T07:01:22.365052+0000","last_fullsized":"2024-09-14T07:01:22.365052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"125'755","reported_seq":6786,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-14T07:01:21.369913+0000","last_change":"2024-09-14T07:01:21.369913+0000","last_active":"2024-09-14T07:01:21.369913+0000","last_peered":"2024-09-14T07:01:21.369913+0000","last_clean":"2024-09-14T07:01:21.369913+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:01:21.369913+0000","last_undegraded":"2024-09-14T07:01:21.369913+0000","last_fullsized":"2024-09-14T07:01:21.369913+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:33:42.659344+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0072874630000000001,"stat_sum":{"num_bytes":6583020,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":19,"num_read":7780,"num_read_kb":8929,"num_write":856,"num_write_kb":5795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":226,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-14T07:01:21.362804+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:01:21.362804+0000","last_peered":"2024-09-14T07:01:21.362804+0000","last_clean":"2024-09-14T07:01:21.362804+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:01:21.362804+0000","last_undegraded":"2024-09-14T07:01:21.362804+0000","last_fullsized":"2024-09-14T07:01:21.362804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"125'467","reported_seq":11216,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-14T07:01:21.481038+0000","last_change":"2024-09-14T07:01:21.372681+0000","last_active":"2024-09-14T07:01:21.481038+0000","last_peered":"2024-09-14T07:01:21.481038+0000","last_clean":"2024-09-14T07:01:21.481038+0000","last_became_active":"2024-09-14T06:59:51.751790+0000","last_became_peered":"2024-09-14T06:59:51.751790+0000","last_unstale":"2024-09-14T07:01:21.481038+0000","last_undegraded":"2024-09-14T07:01:21.481038+0000","last_fullsized":"2024-09-14T07:01:21.481038+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:32.411025+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0096962639999999996,"stat_sum":{"num_bytes":2277376,"num_objects":22,"num_object_clones":1,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":17,"num_read":12378,"num_read_kb":19835,"num_write":453,"num_write_kb":7943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":228,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.365806+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:01:22.365806+0000","last_peered":"2024-09-14T07:01:22.365806+0000","last_clean":"2024-09-14T07:01:22.365806+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:01:22.365806+0000","last_undegraded":"2024-09-14T07:01:22.365806+0000","last_fullsized":"2024-09-14T07:01:22.365806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"121'504","reported_seq":3860,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T07:01:20.360976+0000","last_change":"2024-09-14T07:01:20.361283+0000","last_active":"2024-09-14T07:01:20.360976+0000","last_peered":"2024-09-14T07:01:20.360976+0000","last_clean":"2024-09-14T07:01:20.360976+0000","last_became_active":"2024-09-14T07:00:31.290234+0000","last_became_peered":"2024-09-14T07:00:31.290234+0000","last_unstale":"2024-09-14T07:01:20.360976+0000","last_undegraded":"2024-09-14T07:01:20.360976+0000","last_fullsized":"2024-09-14T07:01:20.360976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:52.087508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015946689999999999,"stat_sum":{"num_bytes":1422950,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":20,"num_read":4047,"num_read_kb":4338,"num_write":426,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"6"}]},{"pgid":"2.4","version":"0'0","reported_seq":242,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T07:01:20.359648+0000","last_change":"2024-09-14T07:00:27.521374+0000","last_active":"2024-09-14T07:01:20.359648+0000","last_peered":"2024-09-14T07:01:20.359648+0000","last_clean":"2024-09-14T07:01:20.359648+0000","last_became_active":"2024-09-14T07:00:27.521101+0000","last_became_peered":"2024-09-14T07:00:27.521101+0000","last_unstale":"2024-09-14T07:01:20.359648+0000","last_undegraded":"2024-09-14T07:01:20.359648+0000","last_fullsized":"2024-09-14T07:01:20.359648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:09:26.896693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"126'561","reported_seq":10120,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.470636+0000","last_change":"2024-09-14T07:01:22.371711+0000","last_active":"2024-09-14T07:01:22.470636+0000","last_peered":"2024-09-14T07:01:22.470636+0000","last_clean":"2024-09-14T07:01:22.470636+0000","last_became_active":"2024-09-14T07:00:31.290358+0000","last_became_peered":"2024-09-14T07:00:31.290358+0000","last_unstale":"2024-09-14T07:01:22.470636+0000","last_undegraded":"2024-09-14T07:01:22.470636+0000","last_fullsized":"2024-09-14T07:01:22.470636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:18:34.790957+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0064894039999999998,"stat_sum":{"num_bytes":24331298,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":21,"num_read":8146,"num_read_kb":12912,"num_write":595,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.3","version":"126'779","reported_seq":8395,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.450890+0000","last_change":"2024-09-14T07:01:22.367223+0000","last_active":"2024-09-14T07:01:22.450890+0000","last_peered":"2024-09-14T07:01:22.450890+0000","last_clean":"2024-09-14T07:01:22.450890+0000","last_became_active":"2024-09-14T07:00:56.623321+0000","last_became_peered":"2024-09-14T07:00:56.623321+0000","last_unstale":"2024-09-14T07:01:22.450890+0000","last_undegraded":"2024-09-14T07:01:22.450890+0000","last_fullsized":"2024-09-14T07:01:22.450890+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":779,"log_dups_size":0,"ondisk_log_size":779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:34.581790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001699576,"stat_sum":{"num_bytes":4167737,"num_objects":25,"num_object_clones":1,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":18,"num_read":8796,"num_read_kb":11092,"num_write":983,"num_write_kb":6130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"2.2","version":"22'2","reported_seq":167,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.372551+0000","last_change":"2024-09-14T07:00:33.531473+0000","last_active":"2024-09-14T07:01:22.372551+0000","last_peered":"2024-09-14T07:01:22.372551+0000","last_clean":"2024-09-14T07:01:22.372551+0000","last_became_active":"2024-09-14T07:00:33.528631+0000","last_became_peered":"2024-09-14T07:00:33.528631+0000","last_unstale":"2024-09-14T07:01:22.372551+0000","last_undegraded":"2024-09-14T07:01:22.372551+0000","last_fullsized":"2024-09-14T07:01:22.372551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:37:51.090265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"124'468","reported_seq":22087,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T07:01:20.360919+0000","last_change":"2024-09-14T07:01:20.360919+0000","last_active":"2024-09-14T07:01:20.360919+0000","last_peered":"2024-09-14T07:01:20.360919+0000","last_clean":"2024-09-14T07:01:20.360919+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:01:20.360919+0000","last_undegraded":"2024-09-14T07:01:20.360919+0000","last_fullsized":"2024-09-14T07:01:20.360919+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:45:36.815109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012149050000000001,"stat_sum":{"num_bytes":4846509,"num_objects":20,"num_object_clones":2,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":13,"num_read":22339,"num_read_kb":44380,"num_write":476,"num_write_kb":17365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"6"}]},{"pgid":"2.1","version":"0'0","reported_seq":232,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T07:01:24.303129+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:01:24.303129+0000","last_peered":"2024-09-14T07:01:24.303129+0000","last_clean":"2024-09-14T07:01:24.303129+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:01:24.303129+0000","last_undegraded":"2024-09-14T07:01:24.303129+0000","last_fullsized":"2024-09-14T07:01:24.303129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"126'368","reported_seq":2437,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.470182+0000","last_change":"2024-09-14T07:01:22.366696+0000","last_active":"2024-09-14T07:01:22.470182+0000","last_peered":"2024-09-14T07:01:22.470182+0000","last_clean":"2024-09-14T07:01:22.470182+0000","last_became_active":"2024-09-14T07:00:30.290337+0000","last_became_peered":"2024-09-14T07:00:30.290337+0000","last_unstale":"2024-09-14T07:01:22.470182+0000","last_undegraded":"2024-09-14T07:01:22.470182+0000","last_fullsized":"2024-09-14T07:01:22.470182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:24:38.400152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015967640000000001,"stat_sum":{"num_bytes":2230860,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":20,"num_read":2999,"num_read_kb":12550,"num_write":299,"num_write_kb":7125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"2.0","version":"0'0","reported_seq":229,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.364887+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T07:01:22.364887+0000","last_peered":"2024-09-14T07:01:22.364887+0000","last_clean":"2024-09-14T07:01:22.364887+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:01:22.364887+0000","last_undegraded":"2024-09-14T07:01:22.364887+0000","last_fullsized":"2024-09-14T07:01:22.364887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"123'684","reported_seq":10554,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-14T07:01:21.362689+0000","last_change":"2024-09-14T07:01:21.363162+0000","last_active":"2024-09-14T07:01:21.362689+0000","last_peered":"2024-09-14T07:01:21.362689+0000","last_clean":"2024-09-14T07:01:21.362689+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:01:21.362689+0000","last_undegraded":"2024-09-14T07:01:21.362689+0000","last_fullsized":"2024-09-14T07:01:21.362689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:36:17.056023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076511799999999996,"stat_sum":{"num_bytes":35306103,"num_objects":36,"num_object_clones":3,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":24,"num_read":11406,"num_read_kb":36576,"num_write":902,"num_write_kb":12556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"b","length":"7"}]},{"pgid":"2.3","version":"20'1","reported_seq":249,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.365140+0000","last_change":"2024-09-14T07:00:27.523385+0000","last_active":"2024-09-14T07:01:22.365140+0000","last_peered":"2024-09-14T07:01:22.365140+0000","last_clean":"2024-09-14T07:01:22.365140+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:01:22.365140+0000","last_undegraded":"2024-09-14T07:01:22.365140+0000","last_fullsized":"2024-09-14T07:01:22.365140+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:19.504174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":319,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.372346+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T07:01:22.372346+0000","last_peered":"2024-09-14T07:01:22.372346+0000","last_clean":"2024-09-14T07:01:22.372346+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:01:22.372346+0000","last_undegraded":"2024-09-14T07:01:22.372346+0000","last_fullsized":"2024-09-14T07:01:22.372346+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"127'590","reported_seq":10011,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T07:01:24.391084+0000","last_change":"2024-09-14T07:01:24.303973+0000","last_active":"2024-09-14T07:01:24.391084+0000","last_peered":"2024-09-14T07:01:24.391084+0000","last_clean":"2024-09-14T07:01:24.391084+0000","last_became_active":"2024-09-14T07:00:53.610102+0000","last_became_peered":"2024-09-14T07:00:53.610102+0000","last_unstale":"2024-09-14T07:01:24.391084+0000","last_undegraded":"2024-09-14T07:01:24.391084+0000","last_fullsized":"2024-09-14T07:01:24.391084+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":590,"log_dups_size":0,"ondisk_log_size":590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:22:49.407867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068851100000000005,"stat_sum":{"num_bytes":1114128,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":17,"num_read":9511,"num_read_kb":20954,"num_write":530,"num_write_kb":10486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4712244,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"7"}]},{"pgid":"3.9","version":"126'723","reported_seq":5497,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.373438+0000","last_change":"2024-09-14T07:01:22.373850+0000","last_active":"2024-09-14T07:01:22.373438+0000","last_peered":"2024-09-14T07:01:22.373438+0000","last_clean":"2024-09-14T07:01:22.373438+0000","last_became_active":"2024-09-14T06:59:28.804973+0000","last_became_peered":"2024-09-14T06:59:28.804973+0000","last_unstale":"2024-09-14T07:01:22.373438+0000","last_undegraded":"2024-09-14T07:01:22.373438+0000","last_fullsized":"2024-09-14T07:01:22.373438+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":723,"log_dups_size":0,"ondisk_log_size":723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:49.777051+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0077330239999999998,"stat_sum":{"num_bytes":2351104,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":17,"num_read":8093,"num_read_kb":17856,"num_write":769,"num_write_kb":7664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.a","version":"126'1102","reported_seq":8301,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.365653+0000","last_change":"2024-09-14T07:01:22.365653+0000","last_active":"2024-09-14T07:01:22.365653+0000","last_peered":"2024-09-14T07:01:22.365653+0000","last_clean":"2024-09-14T07:01:22.365653+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:01:22.365653+0000","last_undegraded":"2024-09-14T07:01:22.365653+0000","last_fullsized":"2024-09-14T07:01:22.365653+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1102,"log_dups_size":0,"ondisk_log_size":1102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:38:34.652637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070506099999999999,"stat_sum":{"num_bytes":1306147,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":17,"num_read":10472,"num_read_kb":9712,"num_write":1518,"num_write_kb":3547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.b","version":"121'338","reported_seq":2808,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-14T07:01:20.360221+0000","last_change":"2024-09-14T07:01:20.360578+0000","last_active":"2024-09-14T07:01:20.360221+0000","last_peered":"2024-09-14T07:01:20.360221+0000","last_clean":"2024-09-14T07:01:20.360221+0000","last_became_active":"2024-09-14T07:00:13.450417+0000","last_became_peered":"2024-09-14T07:00:13.450417+0000","last_unstale":"2024-09-14T07:01:20.360221+0000","last_undegraded":"2024-09-14T07:01:20.360221+0000","last_fullsized":"2024-09-14T07:01:20.360221+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:19:25.431317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084879499999999995,"stat_sum":{"num_bytes":33896278,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":19,"num_read":1933,"num_read_kb":8360,"num_write":281,"num_write_kb":3165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"6"}]},{"pgid":"3.c","version":"128'460","reported_seq":5776,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T07:01:24.419134+0000","last_change":"2024-09-14T07:01:24.303973+0000","last_active":"2024-09-14T07:01:24.419134+0000","last_peered":"2024-09-14T07:01:24.419134+0000","last_clean":"2024-09-14T07:01:24.419134+0000","last_became_active":"2024-09-14T07:00:56.618738+0000","last_became_peered":"2024-09-14T07:00:56.618738+0000","last_unstale":"2024-09-14T07:01:24.419134+0000","last_undegraded":"2024-09-14T07:01:24.419134+0000","last_fullsized":"2024-09-14T07:01:24.419134+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:09.727335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724418,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5158,"num_read_kb":11269,"num_write":417,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"7"}]},{"pgid":"3.d","version":"126'422","reported_seq":2160,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.367040+0000","last_change":"2024-09-14T07:01:22.367040+0000","last_active":"2024-09-14T07:01:22.367040+0000","last_peered":"2024-09-14T07:01:22.367040+0000","last_clean":"2024-09-14T07:01:22.367040+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:01:22.367040+0000","last_undegraded":"2024-09-14T07:01:22.367040+0000","last_fullsized":"2024-09-14T07:01:22.367040+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:21.919569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013564860000000001,"stat_sum":{"num_bytes":9750896,"num_objects":28,"num_object_clones":2,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":22,"num_read":1889,"num_read_kb":6580,"num_write":434,"num_write_kb":6923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.e","version":"126'423","reported_seq":12214,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.372031+0000","last_change":"2024-09-14T07:01:22.372562+0000","last_active":"2024-09-14T07:01:22.372031+0000","last_peered":"2024-09-14T07:01:22.372031+0000","last_clean":"2024-09-14T07:01:22.372031+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:01:22.372031+0000","last_undegraded":"2024-09-14T07:01:22.372031+0000","last_fullsized":"2024-09-14T07:01:22.372031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:47:33.569685+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0063773889999999998,"stat_sum":{"num_bytes":37078617,"num_objects":26,"num_object_clones":2,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":22,"num_read":11918,"num_read_kb":54116,"num_write":405,"num_write_kb":18403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.f","version":"126'503","reported_seq":4767,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.437070+0000","last_change":"2024-09-14T07:01:22.370505+0000","last_active":"2024-09-14T07:01:22.437070+0000","last_peered":"2024-09-14T07:01:22.437070+0000","last_clean":"2024-09-14T07:01:22.437070+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:01:22.437070+0000","last_undegraded":"2024-09-14T07:01:22.437070+0000","last_fullsized":"2024-09-14T07:01:22.437070+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":503,"log_dups_size":0,"ondisk_log_size":503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:05.507756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075240099999999998,"stat_sum":{"num_bytes":5397360,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":28,"num_read":5747,"num_read_kb":11430,"num_write":420,"num_write_kb":4665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.10","version":"125'425","reported_seq":3990,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.372636+0000","last_change":"2024-09-14T07:01:22.373484+0000","last_active":"2024-09-14T07:01:22.372636+0000","last_peered":"2024-09-14T07:01:22.372636+0000","last_clean":"2024-09-14T07:01:22.372636+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:01:22.372636+0000","last_undegraded":"2024-09-14T07:01:22.372636+0000","last_fullsized":"2024-09-14T07:01:22.372636+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:45.097842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001078615,"stat_sum":{"num_bytes":4637112,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":16,"num_read":3603,"num_read_kb":12334,"num_write":388,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.11","version":"126'603","reported_seq":11038,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.369893+0000","last_change":"2024-09-14T07:01:22.370350+0000","last_active":"2024-09-14T07:01:22.369893+0000","last_peered":"2024-09-14T07:01:22.369893+0000","last_clean":"2024-09-14T07:01:22.369893+0000","last_became_active":"2024-09-14T07:00:31.290804+0000","last_became_peered":"2024-09-14T07:00:31.290804+0000","last_unstale":"2024-09-14T07:01:22.369893+0000","last_undegraded":"2024-09-14T07:01:22.369893+0000","last_fullsized":"2024-09-14T07:01:22.369893+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:21:11.167664+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0042560599999999999,"stat_sum":{"num_bytes":7178489,"num_objects":23,"num_object_clones":1,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":12238,"num_read_kb":24167,"num_write":644,"num_write_kb":8977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"8"}]},{"pgid":"3.12","version":"128'701","reported_seq":6764,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-14T07:01:24.418511+0000","last_change":"2024-09-14T07:01:24.304289+0000","last_active":"2024-09-14T07:01:24.418511+0000","last_peered":"2024-09-14T07:01:24.418511+0000","last_clean":"2024-09-14T07:01:24.418511+0000","last_became_active":"2024-09-14T07:00:36.547241+0000","last_became_peered":"2024-09-14T07:00:36.547241+0000","last_unstale":"2024-09-14T07:01:24.418511+0000","last_undegraded":"2024-09-14T07:01:24.418511+0000","last_fullsized":"2024-09-14T07:01:24.418511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:50:56.896467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094106400000000003,"stat_sum":{"num_bytes":26758006,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":20,"num_read":8537,"num_read_kb":13075,"num_write":597,"num_write_kb":4986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"7"}]},{"pgid":"3.13","version":"126'605","reported_seq":6433,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-14T07:01:22.370216+0000","last_change":"2024-09-14T07:01:22.370592+0000","last_active":"2024-09-14T07:01:22.370216+0000","last_peered":"2024-09-14T07:01:22.370216+0000","last_clean":"2024-09-14T07:01:22.370216+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:01:22.370216+0000","last_undegraded":"2024-09-14T07:01:22.370216+0000","last_fullsized":"2024-09-14T07:01:22.370216+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:21:32.612493+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0056748390000000001,"stat_sum":{"num_bytes":22135678,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":6772,"num_read_kb":10244,"num_write":472,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":368147904,"num_objects":784,"num_object_clones":40,"num_object_copies":1568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":784,"num_whiteouts":608,"num_read":239770,"num_read_kb":529424,"num_write":22404,"num_write_kb":271596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2096,"num_bytes_recovered":84516668,"num_keys_recovered":235,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8859648,"data_stored":58008643,"data_compressed":2725145,"data_compressed_allocated":6352896,"data_compressed_original":56619230,"omap_allocated":0,"internal_metadata":0},"log_size":19913,"ondisk_log_size":19913,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":0,"seq":64424509470,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55460,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316380,"statfs":{"total":96636764160,"available":96579973120,"internally_reserved":0,"allocated":2105344,"data_stored":3008726,"data_compressed":226776,"data_compressed_allocated":372736,"data_compressed_original":2157364,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60488,"kb_used_data":2920,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311352,"statfs":{"total":96636764160,"available":96574824448,"internally_reserved":0,"allocated":2990080,"data_stored":11593410,"data_compressed":424933,"data_compressed_allocated":1130496,"data_compressed_original":10670080,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66128,"kb_used_data":3000,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94305712,"statfs":{"total":96636764160,"available":96569049088,"internally_reserved":0,"allocated":3072000,"data_stored":13448704,"data_compressed":436135,"data_compressed_allocated":1241088,"data_compressed_original":12554240,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70792,"kb_used_data":3512,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301048,"statfs":{"total":96636764160,"available":96564273152,"internally_reserved":0,"allocated":3596288,"data_stored":13659270,"data_compressed":677974,"data_compressed_allocated":1503232,"data_compressed_original":12595200,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542173,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34388,"kb_used_data":3200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337452,"statfs":{"total":96636764160,"available":96601550848,"internally_reserved":0,"allocated":3276800,"data_stored":17039852,"data_compressed":589354,"data_compressed_allocated":1359872,"data_compressed_original":16103338,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56328,"kb_used_data":2932,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315512,"statfs":{"total":96636764160,"available":96579084288,"internally_reserved":0,"allocated":3002368,"data_stored":12010089,"data_compressed":429896,"data_compressed_allocated":1150976,"data_compressed_original":11103744,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41388,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330452,"statfs":{"total":96636764160,"available":96594382848,"internally_reserved":0,"allocated":1921024,"data_stored":4342519,"data_compressed":147819,"data_compressed_allocated":258048,"data_compressed_original":3545002,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2076672,"data_stored":12918709,"data_compressed":677904,"data_compressed_allocated":1495040,"data_compressed_original":12578816,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1511424,"data_stored":12140400,"data_compressed":431004,"data_compressed_allocated":1204224,"data_compressed_original":11980800,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1490944,"data_stored":10875184,"data_compressed":424933,"data_compressed_allocated":1130496,"data_compressed_original":10670080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":2282370,"data_compressed":226741,"data_compressed_allocated":368640,"data_compressed_original":2149172,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1343488,"data_stored":9208617,"data_compressed":369292,"data_compressed_allocated":966656,"data_compressed_original":9006592,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1388544,"data_stored":9878938,"data_compressed":446335,"data_compressed_allocated":974848,"data_compressed_original":9664426,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":184320,"data_stored":310283,"data_compressed":48933,"data_compressed_allocated":81920,"data_compressed_original":241664,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":394142,"data_compressed":100003,"data_compressed_allocated":131072,"data_compressed_original":327680,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:01:25.557 INFO:tasks.thrashosds.thrasher:Setting 3.9 to [4, 6] 2024-09-14T07:01:25.558 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.9', '4', '6'] 2024-09-14T07:01:25.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.9 4 6 2024-09-14T07:01:25.627 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:25.626+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:25.650 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:25.650 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:25.659 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:01:25.728 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:25.728+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:25.803 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:25.803 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:25.813 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:01:25.829 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:25.828+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:25.916 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:25.916 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:25.925 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:01:25.930 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:25.929+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:26.028 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:26.028 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:26.031 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:26.031+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:26.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T07:01:26.132 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:26.131+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:26.146 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:26.146 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:26.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T07:01:26.233 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:26.232+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:26.261 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:26.261 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:26.270 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T07:01:26.334 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:26.333+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:26.377 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:26.377 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:26.384 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.9 pg_upmap mapping to [4,6] 2024-09-14T07:01:26.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T07:01:26.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T07:01:26.434 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:26.434+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:26.535 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:26.534+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:26.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T07:01:26.563 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:26.563 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:26.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T07:01:26.657 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:26.656+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:26.686 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:26.686 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:26.704 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:26.704 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:26.720 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:26.721 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:26.725 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:26.725 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:26.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T07:01:26.737 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:26.736+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:26.737 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:26.738 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:26.755 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:26.756 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:26.773 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:26.790 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:26.790 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:26.807 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:26.807 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:26.847 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:26.847 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4204.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.112530787, 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57005258299999995, 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4204", 2024-09-14T07:01:26.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.189:0/2961180613", 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733716+0000", 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733718+0000", 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.559e-06 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733726+0000", 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9219999999999997e-06 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733737+0000", 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1302e-05 2024-09-14T07:01:26.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733748+0000", 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0438999999999999e-05 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733821+0000", 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2929000000000006e-05 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:13.733841+0000", 2024-09-14T07:01:26.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0191e-05 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301316+0000", 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.56747477499999999 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301523+0000", 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00020761099999999999 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-14T07:01:26.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.301677+0000", 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015407600000000001 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303303+0000", 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001626197 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303685+0000", 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038116499999999999 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303721+0000", 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6482000000000001e-05 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:14.303769+0000", 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7935000000000001e-05 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:176998 3.1c 3:3b6a06c0:::rbd_data.114f2b0c87e5.00000000000000af:21 [read 237568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.854 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.954773547999999, 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030101810999999999, 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 176998 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891474+0000", 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1099999999999995e-07 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891479+0000", 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891486+0000", 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7449999999999995e-06 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.891493+0000", 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4829999999999996e-06 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905071+0000", 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013577878999999999 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.905181+0000", 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110005 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.921576+0000", 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016394855 2024-09-14T07:01:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177029 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 14336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.954231903, 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027087837, 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177029 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892015+0000", 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892022+0000", 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7780000000000002e-06 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892026+0000", 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892035+0000", 2024-09-14T07:01:26.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7169999999999997e-06 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892041+0000", 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4620000000000001e-06 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.918935+0000", 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026893675999999998 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919012+0000", 2024-09-14T07:01:26.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7599e-05 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919103+0000", 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0734000000000002e-05 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177030 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 40960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.954188596999998, 2024-09-14T07:01:26.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027219149000000002, 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177030 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:26.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892059+0000", 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3300000000000002e-07 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892062+0000", 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0259999999999999e-06 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892069+0000", 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1409999999999999e-06 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892075+0000", 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5330000000000002e-06 2024-09-14T07:01:26.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919128+0000", 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027053549999999999 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919196+0000", 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7316000000000004e-05 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919278+0000", 2024-09-14T07:01:26.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2050000000000002e-05 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177031 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 67584~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.954157225000003, 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027358328000000001, 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177031 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892090+0000", 2024-09-14T07:01:26.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892091+0000", 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9900000000000001e-07 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892093+0000", 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7180000000000001e-06 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892100+0000", 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7630000000000003e-06 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892106+0000", 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2589999999999997e-06 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919302+0000", 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027195568 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919368+0000", 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6030000000000003e-05 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919448+0000", 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0490999999999999e-05 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.872 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177032 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 94208~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.954125941999997, 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027491581000000001, 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177032 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892121+0000", 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892122+0000", 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892124+0000", 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3010000000000001e-06 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892131+0000", 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8600000000000004e-06 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892137+0000", 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5690000000000004e-06 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919472+0000", 2024-09-14T07:01:26.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027335818000000001 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919539+0000", 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6610000000000001e-05 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919613+0000", 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3930000000000005e-05 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177033 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 120832~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.954101352999999, 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027624723, 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177033 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.878 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892146+0000", 2024-09-14T07:01:26.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892148+0000", 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0889999999999998e-06 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892157+0000", 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9719999999999998e-06 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892163+0000", 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1240000000000004e-06 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919636+0000", 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027473329000000001 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919698+0000", 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2385999999999995e-05 2024-09-14T07:01:26.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919771+0000", 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2384000000000004e-05 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177034 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 147456~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.954076076, 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027770280000000001, 2024-09-14T07:01:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177034 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892171+0000", 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892172+0000", 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892174+0000", 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6620000000000001e-06 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892180+0000", 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0079999999999999e-06 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892185+0000", 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0490000000000003e-06 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919795+0000", 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027610151999999999 2024-09-14T07:01:26.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919864+0000", 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.834e-05 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919941+0000", 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7627999999999996e-05 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177035 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 174080~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.954045247000003, 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027885375, 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177035 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892202+0000", 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892205+0000", 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5239999999999999e-06 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892211+0000", 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3749999999999999e-06 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892217+0000", 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3399999999999997e-06 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.919964+0000", 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027746815000000001 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920025+0000", 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1424e-05 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920087+0000", 2024-09-14T07:01:26.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2438000000000001e-05 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177036 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 200704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.954022956999999, 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028015455000000002, 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177036 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892224+0000", 2024-09-14T07:01:26.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892225+0000", 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2199999999999999e-07 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892227+0000", 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1579999999999999e-06 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892241+0000", 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4353e-05 2024-09-14T07:01:26.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892248+0000", 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4230000000000002e-06 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920111+0000", 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027863849 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920176+0000", 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4873e-05 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920240+0000", 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3377e-05 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177037 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 227328~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.953989524999997, 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028139249000000002, 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177037 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892258+0000", 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892261+0000", 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4889999999999998e-06 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892268+0000", 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7939999999999997e-06 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892290+0000", 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2444999999999999e-05 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920265+0000", 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.027974987 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920332+0000", 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6935999999999997e-05 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920397+0000", 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5081999999999998e-05 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177038 3.1c 3:3ce23852:::rbd_data.114f2b0c87e5.00000000000000b3:21 [read 253952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:26.899 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.95394735, 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028249414, 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177038 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892300+0000", 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2900000000000004e-07 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892303+0000", 2024-09-14T07:01:26.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6790000000000001e-06 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892312+0000", 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2839999999999995e-06 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.892318+0000", 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1460000000000004e-06 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920421+0000", 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.028103197999999999 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920486+0000", 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4960000000000001e-05 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.920549+0000", 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3618e-05 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177325 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.945443484000002, 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026889606999999999, 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177325 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900804+0000", 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3000000000000002e-07 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900808+0000", 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900816+0000", 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4409999999999993e-06 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900820+0000", 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927681+0000", 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026860346 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927693+0000", 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2685e-05 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177326 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:26.907 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.945421947, 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026903018000000001, 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177326 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900825+0000", 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900826+0000", 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900827+0000", 2024-09-14T07:01:26.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.544e-06 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900845+0000", 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7484999999999999e-05 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900849+0000", 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5630000000000002e-06 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927720+0000", 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02687121 2024-09-14T07:01:26.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927728+0000", 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8800000000000008e-06 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177327 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.945391596999997, 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026904081999999999, 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177327 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900856+0000", 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900858+0000", 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.962e-06 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900869+0000", 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.148e-05 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.900874+0000", 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2250000000000002e-06 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927752+0000", 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026878655000000001 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.927760+0000", 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3719999999999999e-06 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177365 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.945036672999997, 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026950818000000001, 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177365 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.915 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901211+0000", 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0100000000000001e-07 2024-09-14T07:01:26.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901212+0000", 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3799999999999999e-06 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901221+0000", 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5809999999999997e-06 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901225+0000", 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928151+0000", 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026925722999999999 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928161+0000", 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0596e-05 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177366 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.945017515000004, 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026960812000000001, 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177366 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:26.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901230+0000", 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.91e-07 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901231+0000", 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.409e-06 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901240+0000", 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5290000000000008e-06 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901244+0000", 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1250000000000003e-06 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928182+0000", 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026937492 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928191+0000", 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9660000000000002e-06 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177367 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:26.922 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.944998255000002, 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026971434999999998, 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177367 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901249+0000", 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901251+0000", 2024-09-14T07:01:26.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6750000000000001e-06 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901259+0000", 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9950000000000005e-06 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901263+0000", 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1119999999999998e-06 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928213+0000", 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026949533000000001 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928220+0000", 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7800000000000001e-06 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177404 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.926 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.944614119000001, 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026884965, 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177404 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901633+0000", 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901635+0000", 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5710000000000001e-06 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901644+0000", 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6969999999999999e-06 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901648+0000", 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0690000000000003e-06 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928511+0000", 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026863267999999999 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928518+0000", 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0600000000000002e-06 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:177405 3.1a 3.7708afda (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.944594629000001, 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026894992, 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 177405 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901653+0000", 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901654+0000", 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901663+0000", 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0280000000000006e-06 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.901668+0000", 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1160000000000001e-06 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928541+0000", 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.026873107 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:49.928548+0000", 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.973e-06 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:26.934 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:26.935 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:26.846+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:26.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T07:01:26.939 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:26.938+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:27.039 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:27.039+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:27.084 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:27.084 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:27.084 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:27.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T07:01:27.141 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:27.140+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:27.200 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:27.200 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:27.200 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:27.200 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:27.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T07:01:27.242 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:27.241+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:27.343 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:27.342+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119746 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1298432~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.811649325000005, 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067798875999999994, 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119746 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545018+0000", 2024-09-14T07:01:27.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545019+0000", 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6699999999999999e-07 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545020+0000", 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.782e-06 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545026+0000", 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9390000000000002e-06 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545031+0000", 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5770000000000004e-06 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612601+0000", 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067570682000000007 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612683+0000", 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1378999999999995e-05 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612817+0000", 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013415000000000001 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119747 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1312768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:27.362 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.811630906000005, 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068036012000000007, 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119747 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.545037+0000", 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546792+0000", 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017554960000000001 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546797+0000", 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1950000000000002e-06 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546808+0000", 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0675e-05 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.546815+0000", 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0079999999999996e-06 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612854+0000", 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.066039158000000001 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.612930+0000", 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5681999999999998e-05 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.613073+0000", 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000142798 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119754 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1413120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.809616206000001, 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067735654000000006, 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119754 2024-09-14T07:01:27.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.547051+0000", 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549973+0000", 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029215999999999999 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549978+0000", 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3510000000000001e-06 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549988+0000", 2024-09-14T07:01:27.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0037000000000001e-05 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.549995+0000", 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9709999999999998e-06 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614569+0000", 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.064573368000000006 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614650+0000", 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1909000000000005e-05 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.614787+0000", 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013641799999999999 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119764 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1556480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.371 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.806444299000006, 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067672107999999995, 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119764 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550223+0000", 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550224+0000", 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550226+0000", 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1720000000000001e-06 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550233+0000", 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9879999999999998e-06 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550238+0000", 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1819999999999997e-06 2024-09-14T07:01:27.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617715+0000", 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067476628999999996 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617807+0000", 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2899999999999995e-05 2024-09-14T07:01:27.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617895+0000", 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7837999999999995e-05 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119765 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:27.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.806421866999997, 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067829500000000001, 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119765 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550246+0000", 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7e-07 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550248+0000", 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1280000000000002e-06 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550256+0000", 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.43e-06 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550261+0000", 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.147e-06 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617926+0000", 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067665280999999994 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.617995+0000", 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9478000000000001e-05 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618075+0000", 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9666000000000006e-05 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119950 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 529408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.806399760999994, 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067992002999999995, 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119950 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550268+0000", 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550852+0000", 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058435400000000004 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550857+0000", 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6979999999999997e-06 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550867+0000", 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0378e-05 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.550874+0000", 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8480000000000003e-06 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618106+0000", 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067231714999999997 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618178+0000", 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2572999999999998e-05 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.618260+0000", 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1437e-05 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119970 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 816128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799744605000001, 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067745828999999994, 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119970 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556923+0000", 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556924+0000", 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0410000000000001e-06 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556927+0000", 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9679999999999998e-06 2024-09-14T07:01:27.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556930+0000", 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624487+0000", 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067557392999999993 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624551+0000", 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3861000000000004e-05 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624669+0000", 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117921 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119971 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 830464~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799734666000006, 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067869065000000006, 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119971 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:27.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556933+0000", 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.22e-07 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556934+0000", 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.096e-06 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556937+0000", 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.796e-06 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556939+0000", 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624708+0000", 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067768992 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624747+0000", 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8148999999999997e-05 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624802+0000", 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.533e-05 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119972 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 844800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799724995000005, 2024-09-14T07:01:27.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068010762000000002, 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119972 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556942+0000", 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556943+0000", 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556944+0000", 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0470000000000001e-06 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556947+0000", 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9720000000000001e-06 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556949+0000", 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4899999999999999e-06 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624838+0000", 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.067888899000000003 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624895+0000", 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7395000000000002e-05 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624953+0000", 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7745999999999999e-05 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119973 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 859136~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799715113000005, 2024-09-14T07:01:27.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068210898000000006, 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119973 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556952+0000", 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556953+0000", 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556954+0000", 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0189999999999999e-06 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556957+0000", 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9490000000000001e-06 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556959+0000", 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3870000000000002e-06 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.624983+0000", 2024-09-14T07:01:27.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068023726000000007 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625049+0000", 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6385999999999998e-05 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625163+0000", 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114221 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119974 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 873472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799705192000005, 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068338594000000003, 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119974 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556962+0000", 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556963+0000", 2024-09-14T07:01:27.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8599999999999999e-07 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556964+0000", 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4920000000000001e-06 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556969+0000", 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.04e-06 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556974+0000", 2024-09-14T07:01:27.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.639e-06 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625194+0000", 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068220742000000001 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625246+0000", 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1844e-05 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625301+0000", 2024-09-14T07:01:27.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4551e-05 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119975 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 887808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799686952000002, 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068554198999999996, 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119975 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:27.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556981+0000", 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7800000000000002e-07 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556983+0000", 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2560000000000001e-06 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556989+0000", 2024-09-14T07:01:27.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2210000000000002e-06 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.556994+0000", 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7779999999999998e-06 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625330+0000", 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068335911999999999 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625396+0000", 2024-09-14T07:01:27.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6051999999999997e-05 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625535+0000", 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013860200000000001 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119976 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 902144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799665957000002, 2024-09-14T07:01:27.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068656164000000006, 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119976 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557002+0000", 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557004+0000", 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3319999999999999e-06 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557010+0000", 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1770000000000003e-06 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557016+0000", 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2650000000000004e-06 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625566+0000", 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068550323999999996 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625604+0000", 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8398000000000001e-05 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625658+0000", 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3239000000000003e-05 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119977 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 916480~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.414 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799585538000002, 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068713522999999999, 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119977 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557082+0000", 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557086+0000", 2024-09-14T07:01:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3569999999999998e-06 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557094+0000", 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6829999999999997e-06 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557101+0000", 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3380000000000001e-06 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625689+0000", 2024-09-14T07:01:27.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068587882000000003 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625743+0000", 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4082000000000002e-05 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625795+0000", 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2657000000000001e-05 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119978 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 930816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799556945999996, 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068826410000000005, 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119978 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557111+0000", 2024-09-14T07:01:27.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6699999999999999e-07 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557113+0000", 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557120+0000", 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1079999999999998e-06 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557125+0000", 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625823+0000", 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068698928000000006 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625884+0000", 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0056999999999998e-05 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625937+0000", 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3383e-05 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119979 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 945152~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799535798999997, 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068940788000000003, 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119979 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557132+0000", 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5199999999999998e-07 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557134+0000", 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.018e-06 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557140+0000", 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999999e-06 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557145+0000", 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7790000000000003e-06 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.625966+0000", 2024-09-14T07:01:27.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068820972999999994 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626019+0000", 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3828e-05 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626072+0000", 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3078000000000002e-05 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119980 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 959488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799515440999997, 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069053630000000005, 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119980 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.428 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557152+0000", 2024-09-14T07:01:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557159+0000", 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557167+0000", 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1149999999999996e-06 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557172+0000", 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626101+0000", 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068928552000000004 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626154+0000", 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3338000000000003e-05 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626206+0000", 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1657999999999999e-05 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119981 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 973824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799486791000007, 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069157629999999998, 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119981 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557181+0000", 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557183+0000", 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557190+0000", 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2249999999999997e-06 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557195+0000", 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3120000000000001e-06 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626234+0000", 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069038926 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626286+0000", 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2604e-05 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626338+0000", 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1947000000000003e-05 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:119982 3.5 3:a5cf371d:::rbd_data.114f480080ee.0000000000000002:14 [read 988160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.799465638000001, 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069364004000000007, 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 119982 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.557202+0000", 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560989+0000", 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037867360000000002 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.560994+0000", 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8989999999999998e-06 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561004+0000", 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3680000000000008e-06 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.561011+0000", 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1640000000000004e-06 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626366+0000", 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.065355286999999998 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626433+0000", 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6554e-05 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:13.626566+0000", 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000132996 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:191378 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99)", 2024-09-14T07:01:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.688378446000002, 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.95875689900000005, 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 191378 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668289+0000", 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668290+0000", 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3319999999999999e-06 2024-09-14T07:01:27.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668295+0000", 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7570000000000004e-06 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668306+0000", 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0813e-05 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668311+0000", 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0839999999999999e-06 2024-09-14T07:01:27.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668364+0000", 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2673999999999997e-05 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:55.668371+0000", 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624394+0000", 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.95602263300000001 2024-09-14T07:01:27.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624558+0000", 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000163525 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.624767+0000", 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000209555 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.626401+0000", 2024-09-14T07:01:27.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016343110000000001 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.626918+0000", 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051661100000000005 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.627005+0000", 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6854999999999994e-05 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:56.627046+0000", 2024-09-14T07:01:27.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0970999999999998e-05 2024-09-14T07:01:27.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.447 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:27.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T07:01:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-14T07:01:27.450 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-14T07:01:27.447+0000 7f92f17c4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33404) UID: 0 2024-09-14T07:01:27.549 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:27.548+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:27.596 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-14T07:01:27.597 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:27.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T07:01:27.651 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:27.650+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:27.751 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:27.750+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:27.753 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:27.753 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-14T07:01:27.753 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-14T07:01:27.753 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-14T07:01:27.753 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:27.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T07:01:27.852 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:27.852+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:27.910 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:27.910 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-14T07:01:27.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4272.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 128.15514997700001, 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36844089699999999, 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4272", 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/1392563886", 2024-09-14T07:01:27.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753798+0000", 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753806+0000", 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.452e-06 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753813+0000", 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9469999999999997e-06 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753826+0000", 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2819999999999999e-05 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753845+0000", 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9785000000000001e-05 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753925+0000", 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9086999999999997e-05 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:19.753932+0000", 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6709999999999999e-06 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120496+0000", 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.36656343000000002 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120696+0000", 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000200161 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.120886+0000", 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019014800000000001 2024-09-14T07:01:27.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.121622+0000", 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073640000000000001 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122136+0000", 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000513764 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122200+0000", 2024-09-14T07:01:27.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3497999999999994e-05 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:20.122239+0000", 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9734000000000003e-05 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2047 3.12 3:4dd46840:::rbd_data.114fe955cd3f.0000000000000003:head [stat,truncate 600296] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.55673738500001, 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93600208200000001, 2024-09-14T07:01:27.917 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2047 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352211+0000", 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352212+0000", 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352218+0000", 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4149999999999998e-06 2024-09-14T07:01:27.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352231+0000", 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-05 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352239+0000", 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8739999999999995e-06 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352306+0000", 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7439000000000003e-05 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.352459+0000", 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015237900000000001 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285503+0000", 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93304425599999996 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285682+0000", 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017925300000000001 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.285821+0000", 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013865000000000001 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288120+0000", 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022992690000000001 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288129+0000", 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2119999999999992e-06 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288163+0000", 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3549e-05 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288213+0000", 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0386000000000002e-05 2024-09-14T07:01:27.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:2051 3.12 3:4b0525d4:::rbd_data.114fe955cd3f.0000000000000006:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.551077742, 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.93078096700000001, 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2051 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357871+0000", 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357872+0000", 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2299999999999999e-07 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357876+0000", 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7299999999999996e-06 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357890+0000", 2024-09-14T07:01:27.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3264999999999999e-05 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357898+0000", 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9580000000000007e-06 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357962+0000", 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4811e-05 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:39.357973+0000", 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0994e-05 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287117+0000", 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.92914364999999999 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287239+0000", 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000121452 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 6", 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.287376+0000", 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000137949 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288431+0000", 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010540759999999999 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:27.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288592+0000", 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016107 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288626+0000", 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4717e-05 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:40.288652+0000", 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5372e-05 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45575 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 508928~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.245241464000003, 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069338842999999997, 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45575 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663707+0000", 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663708+0000", 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663710+0000", 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-06 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663716+0000", 2024-09-14T07:01:27.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663721+0000", 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7589999999999997e-06 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727664+0000", 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.063943481999999996 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.727734+0000", 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9676000000000002e-05 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733046+0000", 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005311947 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45577 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 512000~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.245210671999999, 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069538008999999998, 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45577 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.936 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663738+0000", 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7599999999999997e-07 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663741+0000", 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7e-06 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663748+0000", 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1550000000000001e-06 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663753+0000", 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733076+0000", 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069322781999999999 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733175+0000", 2024-09-14T07:01:27.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9233000000000003e-05 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733276+0000", 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100308 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45579 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 515072~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.245187302000005, 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069708713000000005, 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45579 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:27.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663761+0000", 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663762+0000", 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663763+0000", 2024-09-14T07:01:27.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9350000000000001e-06 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663769+0000", 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9719999999999997e-06 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.663774+0000", 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9609999999999996e-06 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733310+0000", 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069535441000000003 2024-09-14T07:01:27.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733384+0000", 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4237000000000001e-05 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733470+0000", 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5767999999999996e-05 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.944 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45607 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 934912~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.244380793000005, 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069094792000000002, 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45607 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664568+0000", 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1699999999999999e-07 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664570+0000", 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664576+0000", 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9030000000000001e-06 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664581+0000", 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733500+0000", 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918816999999993 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733571+0000", 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0822999999999996e-05 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733662+0000", 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1854000000000005e-05 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45608 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 941056~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.244358399999996, 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076058305000000007, 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45608 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664590+0000", 2024-09-14T07:01:27.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664591+0000", 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664593+0000", 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-14T07:01:27.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664599+0000", 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664604+0000", 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733692+0000", 2024-09-14T07:01:27.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.069087510000000005 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.733760+0000", 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8878e-05 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.740648+0000", 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068878749999999999 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.953 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45610 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 937984~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.244338260000006, 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070579577000000004, 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45610 2024-09-14T07:01:27.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664610+0000", 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664611+0000", 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664613+0000", 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664619+0000", 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2999999999999998e-06 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664624+0000", 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735030+0000", 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070406039000000004 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735100+0000", 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9947e-05 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735190+0000", 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9716999999999995e-05 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45662 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 943104~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.244188105000006, 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070640844999999994, 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45662 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664760+0000", 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664761+0000", 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4799999999999999e-07 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664763+0000", 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.809e-06 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664770+0000", 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2880000000000004e-06 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664775+0000", 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735216+0000", 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070441123999999994 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735312+0000", 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5896999999999993e-05 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735401+0000", 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9491000000000001e-05 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45663 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 946176~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.244132551000007, 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070765474999999994, 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45663 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664816+0000", 2024-09-14T07:01:27.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664817+0000", 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664819+0000", 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664834+0000", 2024-09-14T07:01:27.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4653e-05 2024-09-14T07:01:27.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664840+0000", 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735426+0000", 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070586698000000003 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735496+0000", 2024-09-14T07:01:27.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9066000000000006e-05 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735581+0000", 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5860999999999997e-05 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45664 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 949248~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.244099509999998, 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070921499999999998, 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45664 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664849+0000", 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-14T07:01:27.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664852+0000", 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664858+0000", 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.071e-06 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.664863+0000", 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8339999999999998e-06 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735607+0000", 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.070744049000000003 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735676+0000", 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9314999999999996e-05 2024-09-14T07:01:27.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735770+0000", 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4084000000000002e-05 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45706 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 935936~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.974 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.241746425000002, 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068759527000000001, 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45706 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.975 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667202+0000", 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667205+0000", 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2259999999999999e-06 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667212+0000", 2024-09-14T07:01:27.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667217+0000", 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3730000000000001e-06 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735794+0000", 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068576828000000006 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735867+0000", 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3150000000000003e-05 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735962+0000", 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4574000000000006e-05 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45708 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 939008~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.241723859999993, 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068918739000000007, 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45708 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667225+0000", 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8500000000000002e-07 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667227+0000", 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3149999999999999e-06 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667234+0000", 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7259999999999996e-06 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.667239+0000", 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.735988+0000", 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068748703999999994 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736056+0000", 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8095000000000004e-05 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736143+0000", 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7383e-05 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45775 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 936960~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.240866865000001, 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068811426999999994, 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45775 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668082+0000", 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668084+0000", 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9910000000000001e-06 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668090+0000", 2024-09-14T07:01:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1199999999999999e-06 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.668095+0000", 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.271e-06 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736733+0000", 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068638063999999999 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736801+0000", 2024-09-14T07:01:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7884000000000003e-05 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.736893+0000", 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1707999999999997e-05 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45872 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 956416~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.239649843999999, 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068742639999999994, 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45872 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669299+0000", 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669301+0000", 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9139999999999998e-06 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669307+0000", 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2759999999999997e-06 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669312+0000", 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000002e-06 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737885+0000", 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068573137000000006 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.737954+0000", 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8924999999999995e-05 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738041+0000", 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7149000000000002e-05 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45918 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 958464~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.239522003999994, 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068793067999999999, 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45918 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669426+0000", 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669427+0000", 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669429+0000", 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669435+0000", 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2029999999999997e-06 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669440+0000", 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5859999999999998e-06 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738067+0000", 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068626848000000004 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738134+0000", 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7415000000000004e-05 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738220+0000", 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5576999999999997e-05 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45919 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 964608~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.239501911000005, 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068960745000000004, 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45919 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669447+0000", 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999998e-07 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669449+0000", 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669455+0000", 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2899999999999999e-06 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.669460+0000", 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:28.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738244+0000", 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068783821999999994 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738313+0000", 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9503000000000002e-05 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:28.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.738407+0000", 2024-09-14T07:01:28.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3821999999999997e-05 2024-09-14T07:01:28.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:28.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:28.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:28.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.043 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:45958 3.1d 3:b9c5ded0:::rbd_data.114f54fbf0ed.000000000000000d:9 [read 957440~1024] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected+supports_pool_eio e40)", 2024-09-14T07:01:28.043 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.238475018000003, 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068702634999999998, 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 45958 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670473+0000", 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670474+0000", 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670476+0000", 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1830000000000001e-06 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670482+0000", 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.670487+0000", 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7910000000000004e-06 2024-09-14T07:01:28.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739018+0000", 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.068530878000000003 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739089+0000", 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0691999999999993e-05 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:28.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T06:59:49.739176+0000", 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7403000000000003e-05 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4431.0:138497 3.14 3:298b17d3:::rbd_data.114f74bd3bbf.00000000000002d7:head [set-alloc-hint object_size 262144 write_size 262144,write 218355~43789] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.590967736000003, 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97686345699999999, 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:28.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4431", 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 138497 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317981+0000", 2024-09-14T07:01:28.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.317983+0000", 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1229999999999998e-06 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318003+0000", 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0610000000000001e-05 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-14T07:01:28.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318022+0000", 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8627e-05 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318030+0000", 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7209999999999994e-06 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318101+0000", 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1162999999999996e-05 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:30.318108+0000", 2024-09-14T07:01:28.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291257+0000", 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97314914100000005 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291396+0000", 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013883799999999999 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.291573+0000", 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017716800000000001 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.293147+0000", 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015740400000000001 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294753+0000", 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00160597 2024-09-14T07:01:28.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294792+0000", 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8758999999999999e-05 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-14T07:00:31.294844+0000", 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2188000000000002e-05 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:28.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-14T07:01:28.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-14T07:01:28.055 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:28.056 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T07:01:28.056 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:28.055+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:28.157 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:28.156+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:28.160 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:28.161 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:28.161 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:28.161 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:28.169 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T07:01:28.257 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:28.257+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:28.312 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:28.313 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:28.313 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:28.313 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:28.313 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:28.321 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T07:01:28.358 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:28.358+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:28.459 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-14T07:01:28.459+0000 7f20a0416640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33314) UID: 0 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131358 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983456059999995, 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030805583, 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.483 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131358 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:28.484 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.497999+0000", 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.35e-07 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498000+0000", 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1429999999999999e-06 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498004+0000", 2024-09-14T07:01:28.485 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2569999999999999e-06 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498007+0000", 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.507e-06 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499958+0000", 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001950989 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499962+0000", 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7170000000000002e-06 2024-09-14T07:01:28.486 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528792+0000", 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288306389999999 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528804+0000", 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2096000000000001e-05 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.487 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131359 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983436650000002, 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308363920000001, 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131359 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.488 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498018+0000", 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498019+0000", 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5400000000000002e-07 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.489 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498020+0000", 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4529999999999999e-06 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498023+0000", 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3450000000000002e-06 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498026+0000", 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.661e-06 2024-09-14T07:01:28.490 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.499995+0000", 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00196926 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500002+0000", 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6930000000000001e-06 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528842+0000", 2024-09-14T07:01:28.491 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288401300000001 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528855+0000", 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2595999999999999e-05 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131360 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983425647000004, 2024-09-14T07:01:28.492 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030841943, 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131360 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.493 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498029+0000", 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498030+0000", 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498031+0000", 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1119999999999999e-06 2024-09-14T07:01:28.494 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498034+0000", 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8779999999999998e-06 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498036+0000", 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.599e-06 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500030+0000", 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.001993613 2024-09-14T07:01:28.495 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500034+0000", 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1060000000000002e-06 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528867+0000", 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288327429999999 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528871+0000", 2024-09-14T07:01:28.496 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.673e-06 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131361 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983404902000004, 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308644520000001, 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.497 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131361 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.498 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498050+0000", 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498052+0000", 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.24e-06 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498055+0000", 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1609999999999999e-06 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.499 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498058+0000", 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.05e-06 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500066+0000", 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020081169999999998 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500075+0000", 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 9.0559999999999994e-06 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.500 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528903+0000", 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0288276810000001 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528915+0000", 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1938e-05 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131362 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.501 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983386120999995, 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030860661, 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131362 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.502 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498069+0000", 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.503 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498075+0000", 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5639999999999996e-06 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498079+0000", 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8519999999999997e-06 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498081+0000", 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8660000000000002e-06 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.504 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500101+0000", 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020191219999999999 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500104+0000", 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6890000000000002e-06 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528925+0000", 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028820971 2024-09-14T07:01:28.505 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528930+0000", 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3390000000000003e-06 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131363 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983369873000001, 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030874107, 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.506 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131363 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.507 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498085+0000", 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498086+0000", 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1009999999999999e-06 2024-09-14T07:01:28.508 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498090+0000", 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5609999999999999e-06 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498093+0000", 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7360000000000001e-06 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500136+0000", 2024-09-14T07:01:28.509 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020436030000000002 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500145+0000", 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9530000000000005e-06 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528948+0000", 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028803103 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.510 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528959+0000", 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0808999999999999e-05 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131364 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983353492000006, 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308734129999999, 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.511 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131364 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.512 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498101+0000", 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498102+0000", 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.4999999999999999e-07 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498103+0000", 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.759e-06 2024-09-14T07:01:28.513 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498107+0000", 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.4089999999999999e-06 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498110+0000", 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7760000000000002e-06 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500177+0000", 2024-09-14T07:01:28.514 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020678200000000002 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500183+0000", 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4979999999999997e-06 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528970+0000", 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287867049999999 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528975+0000", 2024-09-14T07:01:28.515 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.1959999999999999e-06 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131365 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983331000999996, 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886024, 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.516 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131365 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.517 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498124+0000", 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498125+0000", 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.17e-06 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498129+0000", 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1640000000000001e-06 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.518 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498131+0000", 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.666e-06 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500210+0000", 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020789559999999999 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500215+0000", 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2549999999999997e-06 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.519 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.528993+0000", 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028777174 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529010+0000", 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.7419999999999999e-05 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131366 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.520 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983312702999996, 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308831409999999, 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131366 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:01:28.521 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498142+0000", 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498143+0000", 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5800000000000001e-07 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.522 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498144+0000", 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.474e-06 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498147+0000", 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.371e-06 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498150+0000", 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6410000000000002e-06 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.523 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500239+0000", 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002088842 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500243+0000", 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6689999999999999e-06 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529020+0000", 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287776420000001 2024-09-14T07:01:28.524 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529025+0000", 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.2440000000000001e-06 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131367 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983301643000004, 2024-09-14T07:01:28.525 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030895135, 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131367 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.526 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498153+0000", 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498154+0000", 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2100000000000001e-07 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498155+0000", 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1310000000000001e-06 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.527 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498157+0000", 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7870000000000002e-06 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498160+0000", 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9009999999999998e-06 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500264+0000", 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021031489999999999 2024-09-14T07:01:28.528 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500268+0000", 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0929999999999996e-06 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529043+0000", 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287752139999999 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529048+0000", 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.6389999999999997e-06 2024-09-14T07:01:28.529 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131368 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983277510999997, 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308865389999999, 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.530 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131368 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498177+0000", 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.531 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498178+0000", 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8200000000000001e-07 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498179+0000", 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.6929999999999999e-06 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498183+0000", 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9439999999999998e-06 2024-09-14T07:01:28.532 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498188+0000", 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9409999999999998e-06 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500287+0000", 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0020982269999999998 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.533 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500290+0000", 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6349999999999999e-06 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529059+0000", 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028768648 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529064+0000", 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.169e-06 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.534 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131369 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983262319000005, 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030886017, 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131369 2024-09-14T07:01:28.535 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.536 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498193+0000", 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.28e-07 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498194+0000", 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1459999999999999e-06 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498197+0000", 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0620000000000001e-06 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.537 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498200+0000", 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6000000000000001e-06 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500310+0000", 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021101840000000002 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500314+0000", 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.126e-06 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.538 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529074+0000", 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.02875961 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529079+0000", 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0610000000000003e-06 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.539 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131370 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983246526000002, 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308930119999999, 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131370 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.540 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498208+0000", 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498209+0000", 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1500000000000001e-07 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.541 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498210+0000", 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.187e-06 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498213+0000", 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.151e-06 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498216+0000", 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8289999999999999e-06 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.542 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500340+0000", 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021244229999999998 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500349+0000", 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8370000000000003e-06 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529096+0000", 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287473309999999 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.543 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529101+0000", 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0390000000000004e-06 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131371 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983228146000002, 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308901880000001, 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.544 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131371 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.545 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498227+0000", 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1400000000000001e-07 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498229+0000", 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1110000000000002e-06 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.546 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498234+0000", 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3070000000000002e-06 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498239+0000", 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.651e-06 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500382+0000", 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021424780000000002 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.547 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500389+0000", 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.5340000000000002e-06 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529112+0000", 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287229410000001 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529117+0000", 2024-09-14T07:01:28.548 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9520000000000002e-06 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131372 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983207030000003, 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030884366, 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.549 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131372 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.550 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498248+0000", 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8599999999999999e-07 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498251+0000", 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2230000000000001e-06 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.551 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498257+0000", 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6370000000000002e-06 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498262+0000", 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500418+0000", 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021561800000000002 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.552 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500423+0000", 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.1520000000000002e-06 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529127+0000", 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0287049150000001 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529132+0000", 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.865e-06 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.553 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131373 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983173966999999, 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308673479999999, 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.554 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131373 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.555 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498281+0000", 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9000000000000002e-07 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498283+0000", 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0829999999999998e-06 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498290+0000", 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1600000000000003e-06 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.556 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498295+0000", 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3759999999999999e-06 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500441+0000", 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021462849999999999 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500445+0000", 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8310000000000003e-06 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.557 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529142+0000", 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028697363 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529148+0000", 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8599999999999998e-06 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131374 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:28.558 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983140351000003, 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308477090000001, 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131374 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:28.559 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498315+0000", 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0900000000000002e-07 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498322+0000", 2024-09-14T07:01:28.560 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5760000000000002e-06 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498328+0000", 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4609999999999996e-06 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498334+0000", 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.7130000000000002e-06 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.561 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500464+0000", 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.002130617 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500468+0000", 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7340000000000002e-06 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529157+0000", 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286893669999999 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.562 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529162+0000", 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.8319999999999996e-06 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131375 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983100214000004, 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308296320000001, 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.563 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131375 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.564 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498355+0000", 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498358+0000", 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.317e-06 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498364+0000", 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5930000000000002e-06 2024-09-14T07:01:28.565 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498369+0000", 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3630000000000002e-06 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500488+0000", 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021187699999999999 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500492+0000", 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.7929999999999999e-06 2024-09-14T07:01:28.566 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529179+0000", 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286873590000001 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529184+0000", 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9819999999999999e-06 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.567 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131376 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983072675000003, 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.030817713, 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131376 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.568 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498382+0000", 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498383+0000", 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.7999999999999996e-07 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.569 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498385+0000", 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.091e-06 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498391+0000", 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1070000000000001e-06 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498396+0000", 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4670000000000003e-06 2024-09-14T07:01:28.570 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500514+0000", 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021177449999999999 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500518+0000", 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8770000000000003e-06 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529195+0000", 2024-09-14T07:01:28.571 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.028676938 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529200+0000", 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.0080000000000002e-06 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:131377 3.5 3.b8ecf3a5 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 71.983033149999997, 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0308008749999999, 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.572 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "reached pg", 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 131377 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:28.573 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498422+0000", 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5499999999999998e-07 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498425+0000", 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.5620000000000002e-06 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498431+0000", 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2539999999999998e-06 2024-09-14T07:01:28.574 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.498436+0000", 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.31e-06 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500539+0000", 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0021025459999999998 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:16.500543+0000", 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.597e-06 2024-09-14T07:01:28.575 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529218+0000", 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0286753930000001 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:17.529223+0000", 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.758e-06 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.576 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.577 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:28.578 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T07:01:28.579 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-14T07:01:28.577+0000 7f3eaa19a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33408) UID: 0 2024-09-14T07:01:28.678 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-14T07:01:28.678+0000 7f986adc8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33396) UID: 0 2024-09-14T07:01:28.721 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:28.721 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:28.721 INFO:teuthology.orchestra.run.smithi189.stdout: "num_ops": 0 2024-09-14T07:01:28.721 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:28.730 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T07:01:28.779 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:28.778+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:28.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T07:01:28.833 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:28.833 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [], 2024-09-14T07:01:28.833 INFO:teuthology.orchestra.run.smithi189.stdout: "complaint_time": 30, 2024-09-14T07:01:28.834 INFO:teuthology.orchestra.run.smithi189.stdout: "num_blocked_ops": 0 2024-09-14T07:01:28.834 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:28.842 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T07:01:28.883 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-14T07:01:28.883+0000 7f4009bbd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33306) UID: 0 2024-09-14T07:01:28.948 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-14T07:01:28.949 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T07:01:28.965 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-14T07:01:28.966 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T07:01:28.980 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:28.979+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:28.982 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-14T07:01:28.982 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: "size": 20, 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 600, 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: "ops": [ 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:15712 3.0 3:0705c9a2:::rbd_data.114f7dc3575d.0000000000000000:8 [read 8347648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 104.82129372, 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044112644999999999, 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.988 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 15712 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.989 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165445+0000", 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165631+0000", 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00018568100000000001 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165634+0000", 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.704e-06 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.990 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165642+0000", 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.0350000000000001e-06 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.165654+0000", 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.1246e-05 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209390+0000", 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043736896999999997 2024-09-14T07:01:28.991 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209479+0000", 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8673999999999998e-05 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T06:59:44.209558+0000", 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.8407999999999999e-05 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.992 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105453 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 415744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.169950615000005, 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044119694000000001, 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105453 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.993 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816788+0000", 2024-09-14T07:01:28.994 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.9999999999999999e-07 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816790+0000", 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.229e-06 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816796+0000", 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9199999999999998e-06 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816800+0000", 2024-09-14T07:01:28.995 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3380000000000001e-06 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860716+0000", 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043916136000000001 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860803+0000", 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7064000000000005e-05 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.996 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860908+0000", 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000104707 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105454 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 890880~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.169935147000004, 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044229220999999999, 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:28.997 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105454 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:28.998 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816803+0000", 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816804+0000", 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.2700000000000001e-07 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816805+0000", 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.283e-06 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:28.999 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816808+0000", 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2210000000000002e-06 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816812+0000", 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.3869999999999999e-06 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860934+0000", 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044122852999999997 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.000 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.860974+0000", 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9396e-05 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861033+0000", 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8854000000000003e-05 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.001 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105455 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 647168~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.169923846000003, 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044438890000000002, 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105455 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.002 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.816815+0000", 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822947+0000", 2024-09-14T07:01:29.003 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0061321070000000004 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822952+0000", 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6770000000000003e-06 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822960+0000", 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.2190000000000004e-06 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.004 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.822966+0000", 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6250000000000001e-06 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861070+0000", 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038103089 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861151+0000", 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.1390000000000005e-05 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.005 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.861254+0000", 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00010278299999999999 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105490 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2025472~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163180373000003, 2024-09-14T07:01:29.006 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044067794, 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105490 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.008 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823558+0000", 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0699999999999999e-07 2024-09-14T07:01:29.009 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823560+0000", 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.08e-06 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823564+0000", 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9949999999999999e-06 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823566+0000", 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7e-06 2024-09-14T07:01:29.010 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867521+0000", 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.043955226 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867567+0000", 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.5170999999999998e-05 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867626+0000", 2024-09-14T07:01:29.011 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9414999999999999e-05 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105491 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2500608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163167768999998, 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044297774999999998, 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.012 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105491 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:29.013 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823571+0000", 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1299999999999999e-07 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823572+0000", 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.052e-06 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.014 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823576+0000", 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2209999999999999e-06 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823579+0000", 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1190000000000001e-06 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867666+0000", 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044086179000000003 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.015 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867754+0000", 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.8398000000000003e-05 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867869+0000", 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00011459300000000001 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.016 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105492 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2244608~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163148253000003, 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044412965999999998, 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105492 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.017 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823590+0000", 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823591+0000", 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.41e-07 2024-09-14T07:01:29.018 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823592+0000", 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.5269999999999999e-06 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823596+0000", 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.6540000000000001e-06 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823599+0000", 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.926e-06 2024-09-14T07:01:29.019 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867900+0000", 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044300705000000003 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.867944+0000", 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4116999999999998e-05 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.020 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868003+0000", 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9695999999999999e-05 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105493 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2246656~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163136211999998, 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044642365000000003, 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.021 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105493 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823602+0000", 2024-09-14T07:01:29.022 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823603+0000", 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0599999999999999e-07 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823604+0000", 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.141e-06 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823610+0000", 2024-09-14T07:01:29.023 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.6680000000000004e-06 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823616+0000", 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.271e-06 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868045+0000", 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044429135000000002 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.024 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868133+0000", 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7666999999999999e-05 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868245+0000", 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000112277 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105494 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2256896~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:29.025 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163115910000002, 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044758026999999999, 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105494 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:29.026 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823623+0000", 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1799999999999999e-07 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823624+0000", 2024-09-14T07:01:29.027 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.2920000000000001e-06 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823629+0000", 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3769999999999998e-06 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823631+0000", 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.8370000000000001e-06 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.028 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868276+0000", 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044644265000000002 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868321+0000", 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4960999999999999e-05 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868381+0000", 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0077000000000001e-05 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.029 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105495 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2258944~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163103594000006, 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044980661999999998, 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105495 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.030 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.031 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823635+0000", 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.1899999999999999e-07 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823636+0000", 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0499999999999999e-06 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823640+0000", 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.0239999999999999e-06 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.032 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823643+0000", 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7149999999999998e-06 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868419+0000", 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044776425000000002 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.033 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868507+0000", 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.7900999999999997e-05 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868616+0000", 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000108328 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105496 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2719744~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:01:29.034 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163092153999997, 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045102077999999997, 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105496 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.035 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823646+0000", 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823647+0000", 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0900000000000001e-07 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823648+0000", 2024-09-14T07:01:29.036 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.375e-06 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823651+0000", 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1470000000000001e-06 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823654+0000", 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.6769999999999999e-06 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.037 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868645+0000", 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.04499098 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868689+0000", 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4286000000000003e-05 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868749+0000", 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9404000000000002e-05 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.038 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105497 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2721792~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163081654999999, 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045327083999999997, 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.039 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105497 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.040 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823657+0000", 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0800000000000001e-07 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823658+0000", 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.125e-06 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823661+0000", 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.044e-06 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.041 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823664+0000", 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7020000000000001e-06 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868787+0000", 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045122965000000001 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868877+0000", 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.9650999999999999e-05 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.042 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.868984+0000", 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000107389 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105498 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2478080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163071462000005, 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045455171000000003, 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.043 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105498 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.044 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823667+0000", 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.41e-07 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823669+0000", 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.128e-06 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.045 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823672+0000", 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.2720000000000002e-06 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823676+0000", 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.2259999999999999e-06 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869015+0000", 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045339192 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.046 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869060+0000", 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4370999999999999e-05 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869122+0000", 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.2741000000000001e-05 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105499 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2480128~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:29.047 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163055915000001, 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045567041000000003, 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105499 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:29.048 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823683+0000", 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4000000000000002e-07 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823685+0000", 2024-09-14T07:01:29.049 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.933e-06 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823699+0000", 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.4041e-05 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823705+0000", 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.5990000000000001e-06 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.050 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869146+0000", 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045441606000000002 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869190+0000", 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3927000000000001e-05 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869250+0000", 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9395000000000003e-05 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.051 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105500 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2734080~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163026306999996, 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045695725999999999, 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105500 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.052 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823712+0000", 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823713+0000", 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9400000000000001e-07 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.053 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823715+0000", 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.9539999999999998e-06 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823722+0000", 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 7.1230000000000002e-06 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823728+0000", 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.1619999999999996e-06 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.054 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869286+0000", 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045557687999999999 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869339+0000", 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3526999999999998e-05 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869408+0000", 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.8878e-05 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.055 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105501 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2732032~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.163004657000002, 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045803112999999999, 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.056 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105501 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823734+0000", 2024-09-14T07:01:29.057 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.34e-07 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823736+0000", 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.0660000000000002e-06 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823743+0000", 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.9099999999999999e-06 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.058 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823748+0000", 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.9980000000000003e-06 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869434+0000", 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045685735999999998 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869478+0000", 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3392000000000001e-05 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.059 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869537+0000", 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9577000000000002e-05 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105502 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 2713600~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.162982417999999, 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.045911181000000002, 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.060 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105502 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.061 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.823756+0000", 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828928+0000", 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0051713180000000003 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828931+0000", 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.9419999999999997e-06 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.062 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828940+0000", 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 8.3860000000000007e-06 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.828945+0000", 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.6630000000000001e-06 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869564+0000", 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.040619293000000001 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.063 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869608+0000", 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.4532e-05 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.869667+0000", 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105531 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 894976~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:29.064 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.157386052999996, 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044175939999999997, 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105531 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.065 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829353+0000", 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.04e-07 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829354+0000", 2024-09-14T07:01:29.066 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0720000000000001e-06 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829357+0000", 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.0879999999999999e-06 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829360+0000", 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 2.7080000000000002e-06 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873426+0000", 2024-09-14T07:01:29.067 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044066613999999997 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873470+0000", 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3801000000000003e-05 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873529+0000", 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.8452999999999998e-05 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.068 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:105532 3.0 3:015b325e:::rbd_data.114f40eb775b.0000000000000000:d [read 1136640~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 81.157375999999999, 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.044289753000000001, 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "started", 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 105532 2024-09-14T07:01:29.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.829363+0000", 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834977+0000", 2024-09-14T07:01:29.070 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.0056139579999999996 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834980+0000", 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.8399999999999997e-06 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834991+0000", 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0764999999999999e-05 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.071 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.834998+0000", 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.4570000000000002e-06 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873554+0000", 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.038556422999999999 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873598+0000", 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 4.3622999999999999e-05 2024-09-14T07:01:29.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:07.873652+0000", 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.4687e-05 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: "description": "osd_op(client.4431.0:118426 3.b 3:d50c2793:::rbd_data.114f74d20483.000000000000001c:head [stat,delete] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: "initiated_at": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: "age": 76.485994196999997, 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.95135023100000005, 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: "continuous": false, 2024-09-14T07:01:29.073 INFO:teuthology.orchestra.run.smithi189.stdout: "type_data": { 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "client_info": { 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "client": "client.4431", 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "client_addr": "172.21.15.7:0/2179352736", 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "tid": 118426 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "events": [ 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "initiated", 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "header_read", 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500744+0000", 2024-09-14T07:01:29.074 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "throttled", 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500745+0000", 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.7700000000000004e-07 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "all_read", 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500748+0000", 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.1329999999999999e-06 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "dispatched", 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500759+0000", 2024-09-14T07:01:29.075 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 1.0475000000000001e-05 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "queued_for_pg", 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500765+0000", 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.0430000000000004e-06 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500824+0000", 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.9046999999999999e-05 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for peered", 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:12.500830+0000", 2024-09-14T07:01:29.076 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 6.5889999999999999e-06 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "reached_pg", 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450787+0000", 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.94995644099999998 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "started", 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.450901+0000", 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.000114032 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "waiting for subops from 0", 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451052+0000", 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00015113 2024-09-14T07:01:29.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "op_commit", 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.451746+0000", 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00069442799999999997 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "sub_op_commit_rec", 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452006+0000", 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 0.00025975 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "commit_sent", 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452059+0000", 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 5.3090000000000002e-05 2024-09-14T07:01:29.078 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout: { 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout: "event": "done", 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout: "time": "2024-09-14T07:00:13.452095+0000", 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout: "duration": 3.5395999999999998e-05 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-09-14T07:01:29.079 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-14T07:01:29.080 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-14T07:01:29.080 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T07:01:29.081 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-14T07:01:29.081 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T07:01:29.081 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T07:01:29.082 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-14T07:01:29.082 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T07:01:29.082 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6: {} 2024-09-14T07:01:29.082 INFO:teuthology.orchestra.run.smithi007.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T07:01:29.083 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7: {} 2024-09-14T07:01:29.083 INFO:teuthology.orchestra.run.smithi007.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T07:01:29.122 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:29.121+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:29.185 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-14T07:01:29.184+0000 7f7fd1967640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33387) UID: 0 2024-09-14T07:01:29.225 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:29.225 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:29.234 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T07:01:29.285 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-14T07:01:29.285+0000 7f4082088640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33307) UID: 0 2024-09-14T07:01:29.341 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:29.341 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:29.350 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T07:01:29.379 INFO:teuthology.orchestra.run.smithi007.stdout:All operations completed A-OK! 2024-09-14T07:01:29.390 DEBUG:teuthology.parallel:result is None 2024-09-14T07:01:29.390 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-14T07:01:29.399 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-14T07:01:29.442 INFO:tasks.thrashosds:joining thrashosds 2024-09-14T07:01:29.443 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-14T07:01:29.391+0000 7fdeb8ec2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33312) UID: 0 2024-09-14T07:01:29.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T07:01:29.547 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:29.547 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:29.556 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T07:01:29.659 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:29.659 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:29.668 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T07:01:29.771 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:29.771 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:29.780 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T07:01:29.883 INFO:teuthology.orchestra.run.smithi189.stdout:ERROR: (22) Invalid argument 2024-09-14T07:01:29.883 INFO:teuthology.orchestra.run.smithi189.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T07:01:30.403 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-14T07:01:30.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T07:01:31.401 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-14T07:01:31.404 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-14T07:01:31.404 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-14T07:01:31.404 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-14T07:01:31.722 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-14T07:01:31.721+0000 7fbb91e53640 -1 mon.a@0(leader).osd e135 definitely_dead 0 2024-09-14T07:01:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.6. 2024-09-14T07:01:32.428 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-14T07:01:32.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-14T07:01:32.550 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a: {} 2024-09-14T07:01:32.551 INFO:teuthology.orchestra.run.smithi007.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T07:01:32.561 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b: {} 2024-09-14T07:01:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T07:01:32.570 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c: {} 2024-09-14T07:01:32.570 INFO:teuthology.orchestra.run.smithi007.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T07:01:32.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T07:01:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T07:01:33.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T07:01:34.298 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T07:01:34.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T07:01:35.298 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T07:01:35.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-14T07:01:36.299 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-14T07:01:36.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T07:01:37.297 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T07:01:37.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T07:01:38.433 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T07:01:38.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T07:01:39.436 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T07:01:39.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T07:01:40.442 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T07:01:40.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T07:01:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T07:01:41.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T07:01:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T07:01:42.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T07:01:43.456 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T07:01:43.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-14T07:01:44.296 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-14T07:01:44.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T07:01:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T07:01:45.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-14T07:01:46.478 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-14T07:01:46.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T07:01:47.484 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T07:01:47.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-14T07:01:48.490 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-14T07:01:48.508 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-14T07:01:48.508 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-14T07:01:48.508 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-14T07:01:48.508 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-14T07:01:48.509 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T07:01:48.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T07:01:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:48.815 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"fsid":"32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6","created":"2024-09-14T06:58:59.479055+0000","modified":"2024-09-14T07:01:48.485101+0000","last_up_change":"2024-09-14T06:59:11.718934+0000","last_in_change":"2024-09-14T07:01:32.401522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:59:11.765419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:59:18.343475+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T06:59:27.257805+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":36,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c036038d-cffb-4584-8851-69604a31ee40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6817","nonce":2081065366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6819","nonce":2081065366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6823","nonce":2081065366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6821","nonce":2081065366}]},"public_addr":"172.21.15.7:6817/2081065366","cluster_addr":"172.21.15.7:6819/2081065366","heartbeat_back_addr":"172.21.15.7:6823/2081065366","heartbeat_front_addr":"172.21.15.7:6821/2081065366","state":["exists","up"]},{"osd":1,"uuid":"1c80080f-932a-4957-b64b-f30c4bdb9340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6806","nonce":1688050909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6810","nonce":1688050909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6815","nonce":1688050909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6813","nonce":1688050909}]},"public_addr":"172.21.15.7:6806/1688050909","cluster_addr":"172.21.15.7:6810/1688050909","heartbeat_back_addr":"172.21.15.7:6815/1688050909","heartbeat_front_addr":"172.21.15.7:6813/1688050909","state":["exists","up"]},{"osd":2,"uuid":"f3544d11-8a0f-498c-a03c-5d34134c9773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6825","nonce":1872259061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6827","nonce":1872259061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6831","nonce":1872259061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6829","nonce":1872259061}]},"public_addr":"172.21.15.7:6825/1872259061","cluster_addr":"172.21.15.7:6827/1872259061","heartbeat_back_addr":"172.21.15.7:6831/1872259061","heartbeat_front_addr":"172.21.15.7:6829/1872259061","state":["exists","up"]},{"osd":3,"uuid":"d2261455-c91b-414b-925f-388f0e094dbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6801","nonce":1193301030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6803","nonce":1193301030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6811","nonce":1193301030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6807","nonce":1193301030}]},"public_addr":"172.21.15.7:6801/1193301030","cluster_addr":"172.21.15.7:6803/1193301030","heartbeat_back_addr":"172.21.15.7:6811/1193301030","heartbeat_front_addr":"172.21.15.7:6807/1193301030","state":["exists","up"]},{"osd":4,"uuid":"4088d23b-0d06-4415-9eba-6fb64355746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6811","nonce":2283562461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6813","nonce":2283562461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6817","nonce":2283562461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6815","nonce":2283562461}]},"public_addr":"172.21.15.189:6811/2283562461","cluster_addr":"172.21.15.189:6813/2283562461","heartbeat_back_addr":"172.21.15.189:6817/2283562461","heartbeat_front_addr":"172.21.15.189:6815/2283562461","state":["exists","up"]},{"osd":5,"uuid":"5e27cb53-ecce-4061-bc5a-be599bf18cc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6819","nonce":2212769626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6821","nonce":2212769626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6825","nonce":2212769626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6823","nonce":2212769626}]},"public_addr":"172.21.15.189:6819/2212769626","cluster_addr":"172.21.15.189:6821/2212769626","heartbeat_back_addr":"172.21.15.189:6825/2212769626","heartbeat_front_addr":"172.21.15.189:6823/2212769626","state":["exists","up"]},{"osd":6,"uuid":"ceb9a1f1-f54c-48d5-91d8-2287cac96ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6827","nonce":3197231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6829","nonce":3197231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6833","nonce":3197231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6831","nonce":3197231389}]},"public_addr":"172.21.15.189:6827/3197231389","cluster_addr":"172.21.15.189:6829/3197231389","heartbeat_back_addr":"172.21.15.189:6833/3197231389","heartbeat_front_addr":"172.21.15.189:6831/3197231389","state":["exists","up"]},{"osd":7,"uuid":"85c59cd6-6e2b-4cc0-852d-d1f075a8826d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6801","nonce":2162534312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6803","nonce":2162534312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6807","nonce":2162534312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6805","nonce":2162534312}]},"public_addr":"172.21.15.189:6801/2162534312","cluster_addr":"172.21.15.189:6803/2162534312","heartbeat_back_addr":"172.21.15.189:6807/2162534312","heartbeat_front_addr":"172.21.15.189:6805/2162534312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.064941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.780598+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.127700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.730995+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.864423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.329361+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:10.107581+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.300043+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.8","osds":[7,0]},{"pgid":"3.9","osds":[4,6]},{"pgid":"3.11","osds":[6,7]},{"pgid":"3.18","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":7,"to":5}]},{"pgid":"3.1f","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:01:48.828 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T07:01:48.828 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T07:01:49.129 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:49.129 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"fsid":"32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6","created":"2024-09-14T06:58:59.479055+0000","modified":"2024-09-14T07:01:48.485101+0000","last_up_change":"2024-09-14T06:59:11.718934+0000","last_in_change":"2024-09-14T07:01:32.401522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:59:11.765419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:59:18.343475+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T06:59:27.257805+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":36,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c036038d-cffb-4584-8851-69604a31ee40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6817","nonce":2081065366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6819","nonce":2081065366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6823","nonce":2081065366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6821","nonce":2081065366}]},"public_addr":"172.21.15.7:6817/2081065366","cluster_addr":"172.21.15.7:6819/2081065366","heartbeat_back_addr":"172.21.15.7:6823/2081065366","heartbeat_front_addr":"172.21.15.7:6821/2081065366","state":["exists","up"]},{"osd":1,"uuid":"1c80080f-932a-4957-b64b-f30c4bdb9340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6806","nonce":1688050909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6810","nonce":1688050909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6815","nonce":1688050909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6813","nonce":1688050909}]},"public_addr":"172.21.15.7:6806/1688050909","cluster_addr":"172.21.15.7:6810/1688050909","heartbeat_back_addr":"172.21.15.7:6815/1688050909","heartbeat_front_addr":"172.21.15.7:6813/1688050909","state":["exists","up"]},{"osd":2,"uuid":"f3544d11-8a0f-498c-a03c-5d34134c9773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6825","nonce":1872259061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6827","nonce":1872259061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6831","nonce":1872259061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6829","nonce":1872259061}]},"public_addr":"172.21.15.7:6825/1872259061","cluster_addr":"172.21.15.7:6827/1872259061","heartbeat_back_addr":"172.21.15.7:6831/1872259061","heartbeat_front_addr":"172.21.15.7:6829/1872259061","state":["exists","up"]},{"osd":3,"uuid":"d2261455-c91b-414b-925f-388f0e094dbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6801","nonce":1193301030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6803","nonce":1193301030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6811","nonce":1193301030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6807","nonce":1193301030}]},"public_addr":"172.21.15.7:6801/1193301030","cluster_addr":"172.21.15.7:6803/1193301030","heartbeat_back_addr":"172.21.15.7:6811/1193301030","heartbeat_front_addr":"172.21.15.7:6807/1193301030","state":["exists","up"]},{"osd":4,"uuid":"4088d23b-0d06-4415-9eba-6fb64355746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6811","nonce":2283562461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6813","nonce":2283562461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6817","nonce":2283562461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6815","nonce":2283562461}]},"public_addr":"172.21.15.189:6811/2283562461","cluster_addr":"172.21.15.189:6813/2283562461","heartbeat_back_addr":"172.21.15.189:6817/2283562461","heartbeat_front_addr":"172.21.15.189:6815/2283562461","state":["exists","up"]},{"osd":5,"uuid":"5e27cb53-ecce-4061-bc5a-be599bf18cc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6819","nonce":2212769626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6821","nonce":2212769626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6825","nonce":2212769626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6823","nonce":2212769626}]},"public_addr":"172.21.15.189:6819/2212769626","cluster_addr":"172.21.15.189:6821/2212769626","heartbeat_back_addr":"172.21.15.189:6825/2212769626","heartbeat_front_addr":"172.21.15.189:6823/2212769626","state":["exists","up"]},{"osd":6,"uuid":"ceb9a1f1-f54c-48d5-91d8-2287cac96ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6827","nonce":3197231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6829","nonce":3197231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6833","nonce":3197231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6831","nonce":3197231389}]},"public_addr":"172.21.15.189:6827/3197231389","cluster_addr":"172.21.15.189:6829/3197231389","heartbeat_back_addr":"172.21.15.189:6833/3197231389","heartbeat_front_addr":"172.21.15.189:6831/3197231389","state":["exists","up"]},{"osd":7,"uuid":"85c59cd6-6e2b-4cc0-852d-d1f075a8826d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6801","nonce":2162534312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6803","nonce":2162534312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6807","nonce":2162534312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6805","nonce":2162534312}]},"public_addr":"172.21.15.189:6801/2162534312","cluster_addr":"172.21.15.189:6803/2162534312","heartbeat_back_addr":"172.21.15.189:6807/2162534312","heartbeat_front_addr":"172.21.15.189:6805/2162534312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.064941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.780598+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.127700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.730995+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.864423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.329361+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:10.107581+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.300043+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.8","osds":[7,0]},{"pgid":"3.9","osds":[4,6]},{"pgid":"3.11","osds":[6,7]},{"pgid":"3.18","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":7,"to":5}]},{"pgid":"3.1f","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:01:49.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T07:01:49.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T07:01:49.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T07:01:49.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T07:01:49.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T07:01:49.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T07:01:49.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T07:01:49.145 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T07:01:49.396 INFO:teuthology.orchestra.run.smithi007.stdout:55834574883 2024-09-14T07:01:49.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T07:01:49.399 INFO:teuthology.orchestra.run.smithi007.stdout:64424509476 2024-09-14T07:01:49.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T07:01:49.400 INFO:teuthology.orchestra.run.smithi007.stdout:64424509475 2024-09-14T07:01:49.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T07:01:49.401 INFO:teuthology.orchestra.run.smithi007.stdout:64424509475 2024-09-14T07:01:49.402 INFO:teuthology.orchestra.run.smithi007.stdout:64424509475 2024-09-14T07:01:49.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T07:01:49.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T07:01:49.404 INFO:teuthology.orchestra.run.smithi007.stdout:60129542179 2024-09-14T07:01:49.404 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T07:01:49.405 INFO:teuthology.orchestra.run.smithi007.stdout:60129542179 2024-09-14T07:01:49.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T07:01:49.406 INFO:teuthology.orchestra.run.smithi007.stdout:64424509476 2024-09-14T07:01:49.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T07:01:49.977 INFO:teuthology.orchestra.run.smithi007.stdout:55834574882 2024-09-14T07:01:49.993 INFO:teuthology.orchestra.run.smithi007.stdout:64424509475 2024-09-14T07:01:49.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574882 for osd.7 2024-09-14T07:01:50.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509475 for osd.3 2024-09-14T07:01:50.012 INFO:teuthology.orchestra.run.smithi007.stdout:64424509474 2024-09-14T07:01:50.032 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509474 for osd.0 2024-09-14T07:01:50.037 INFO:teuthology.orchestra.run.smithi007.stdout:64424509474 2024-09-14T07:01:50.042 INFO:teuthology.orchestra.run.smithi007.stdout:60129542178 2024-09-14T07:01:50.043 INFO:teuthology.orchestra.run.smithi007.stdout:64424509474 2024-09-14T07:01:50.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509474 for osd.2 2024-09-14T07:01:50.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542178 for osd.5 2024-09-14T07:01:50.060 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509474 for osd.1 2024-09-14T07:01:50.073 INFO:teuthology.orchestra.run.smithi007.stdout:60129542178 2024-09-14T07:01:50.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542178 for osd.4 2024-09-14T07:01:50.101 INFO:teuthology.orchestra.run.smithi007.stdout:64424509475 2024-09-14T07:01:50.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509475 for osd.6 2024-09-14T07:01:50.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T07:01:51.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T07:01:51.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T07:01:51.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T07:01:51.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T07:01:51.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T07:01:51.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T07:01:51.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T07:01:51.561 INFO:teuthology.orchestra.run.smithi007.stdout:55834574884 2024-09-14T07:01:51.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574883 got 55834574884 for osd.7 2024-09-14T07:01:51.581 DEBUG:teuthology.parallel:result is None 2024-09-14T07:01:51.632 INFO:teuthology.orchestra.run.smithi007.stdout:64424509476 2024-09-14T07:01:51.635 INFO:teuthology.orchestra.run.smithi007.stdout:60129542179 2024-09-14T07:01:51.650 INFO:teuthology.orchestra.run.smithi007.stdout:64424509475 2024-09-14T07:01:51.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542179 for osd.5 2024-09-14T07:01:51.651 DEBUG:teuthology.parallel:result is None 2024-09-14T07:01:51.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509476 for osd.3 2024-09-14T07:01:51.652 DEBUG:teuthology.parallel:result is None 2024-09-14T07:01:51.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509475 for osd.0 2024-09-14T07:01:51.664 DEBUG:teuthology.parallel:result is None 2024-09-14T07:01:51.710 INFO:teuthology.orchestra.run.smithi007.stdout:64424509475 2024-09-14T07:01:51.721 INFO:teuthology.orchestra.run.smithi007.stdout:60129542179 2024-09-14T07:01:51.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509475 for osd.2 2024-09-14T07:01:51.724 DEBUG:teuthology.parallel:result is None 2024-09-14T07:01:51.727 INFO:teuthology.orchestra.run.smithi007.stdout:64424509475 2024-09-14T07:01:51.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542179 got 60129542179 for osd.4 2024-09-14T07:01:51.734 DEBUG:teuthology.parallel:result is None 2024-09-14T07:01:51.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509475 got 64424509475 for osd.1 2024-09-14T07:01:51.745 DEBUG:teuthology.parallel:result is None 2024-09-14T07:01:51.754 INFO:teuthology.orchestra.run.smithi007.stdout:64424509476 2024-09-14T07:01:51.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509476 got 64424509476 for osd.6 2024-09-14T07:01:51.768 DEBUG:teuthology.parallel:result is None 2024-09-14T07:01:51.768 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T07:01:51.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:01:52.108 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:52.109 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:01:52.120 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-14T07:01:51.777824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":575,"num_read":241606,"num_read_kb":531007,"num_write":22900,"num_write_kb":273313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3193,"num_bytes_recovered":84516725,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20502,"ondisk_log_size":20502,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":431780,"kb_used_data":14300,"kb_used_omap":12,"kb_used_meta":417267,"kb_avail":754542940,"statfs":{"total":773094113280,"available":772651970560,"internally_reserved":0,"allocated":14643200,"data_stored":8188451,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":427281969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001513"},"pg_stats":[{"pgid":"3.1f","version":"133'538","reported_seq":7941,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489581+0000","last_change":"2024-09-14T07:01:29.346010+0000","last_active":"2024-09-14T07:01:48.489581+0000","last_peered":"2024-09-14T07:01:48.489581+0000","last_clean":"2024-09-14T07:01:48.489581+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:01:48.489581+0000","last_undegraded":"2024-09-14T07:01:48.489581+0000","last_fullsized":"2024-09-14T07:01:48.489581+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:00.019940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099215999999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8424,"num_read_kb":37327,"num_write":508,"num_write_kb":21659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"130'423","reported_seq":3021,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489455+0000","last_change":"2024-09-14T07:01:29.367999+0000","last_active":"2024-09-14T07:01:48.489455+0000","last_peered":"2024-09-14T07:01:48.489455+0000","last_clean":"2024-09-14T07:01:48.489455+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:01:48.489455+0000","last_undegraded":"2024-09-14T07:01:48.489455+0000","last_fullsized":"2024-09-14T07:01:48.489455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:38.115466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001057501,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2716,"num_read_kb":8402,"num_write":360,"num_write_kb":7998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"130'339","reported_seq":5823,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:43.465257+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T07:01:43.462852+0000","last_became_peered":"2024-09-14T07:01:43.462852+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:36.409964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000797209,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5182,"num_read_kb":12861,"num_write":295,"num_write_kb":5166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"133'2042","reported_seq":4848,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116619+0000","last_change":"2024-09-14T07:01:29.296724+0000","last_active":"2024-09-14T07:01:49.116619+0000","last_peered":"2024-09-14T07:01:49.116619+0000","last_clean":"2024-09-14T07:01:49.116619+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:01:49.116619+0000","last_undegraded":"2024-09-14T07:01:49.116619+0000","last_fullsized":"2024-09-14T07:01:49.116619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":2042,"log_dups_size":0,"ondisk_log_size":2042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:44.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096933700000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11671,"num_read_kb":22543,"num_write":3453,"num_write_kb":11219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"131'900","reported_seq":3379,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116967+0000","last_change":"2024-09-14T07:01:29.299798+0000","last_active":"2024-09-14T07:01:49.116967+0000","last_peered":"2024-09-14T07:01:49.116967+0000","last_clean":"2024-09-14T07:01:49.116967+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:01:49.116967+0000","last_undegraded":"2024-09-14T07:01:49.116967+0000","last_fullsized":"2024-09-14T07:01:49.116967+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:51.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087443700000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4436,"num_read_kb":12231,"num_write":1137,"num_write_kb":20066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"133'1187","reported_seq":8951,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116584+0000","last_change":"2024-09-14T07:01:29.296969+0000","last_active":"2024-09-14T07:01:49.116584+0000","last_peered":"2024-09-14T07:01:49.116584+0000","last_clean":"2024-09-14T07:01:49.116584+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:01:49.116584+0000","last_undegraded":"2024-09-14T07:01:49.116584+0000","last_fullsized":"2024-09-14T07:01:49.116584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:26:48.784038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015092009999999999,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":13058,"num_read_kb":21088,"num_write":1639,"num_write_kb":6895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"133'517","reported_seq":5015,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.300460+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:54:21.579346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00137473,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4742,"num_read_kb":12493,"num_write":520,"num_write_kb":11830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"132'662","reported_seq":3807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:29.346100+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:15.330383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011507799999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5024,"num_read_kb":10270,"num_write":682,"num_write_kb":6392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"130'392","reported_seq":6847,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489521+0000","last_change":"2024-09-14T07:01:29.368091+0000","last_active":"2024-09-14T07:01:48.489521+0000","last_peered":"2024-09-14T07:01:48.489521+0000","last_clean":"2024-09-14T07:01:48.489521+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:01:48.489521+0000","last_undegraded":"2024-09-14T07:01:48.489521+0000","last_fullsized":"2024-09-14T07:01:48.489521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:17.259452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010952469999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7606,"num_read_kb":16770,"num_write":322,"num_write_kb":2846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'413","reported_seq":5194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116357+0000","last_change":"2024-09-14T07:01:43.464245+0000","last_active":"2024-09-14T07:01:49.116357+0000","last_peered":"2024-09-14T07:01:49.116357+0000","last_clean":"2024-09-14T07:01:49.116357+0000","last_became_active":"2024-09-14T07:01:43.462372+0000","last_became_peered":"2024-09-14T07:01:43.462372+0000","last_unstale":"2024-09-14T07:01:49.116357+0000","last_undegraded":"2024-09-14T07:01:49.116357+0000","last_fullsized":"2024-09-14T07:01:49.116357+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:32:17.252003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022563269999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4902,"num_read_kb":8441,"num_write":353,"num_write_kb":3835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"130'346","reported_seq":2783,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489545+0000","last_change":"2024-09-14T07:01:36.306565+0000","last_active":"2024-09-14T07:01:48.489545+0000","last_peered":"2024-09-14T07:01:48.489545+0000","last_clean":"2024-09-14T07:01:48.489545+0000","last_became_active":"2024-09-14T07:01:36.304931+0000","last_became_peered":"2024-09-14T07:01:36.304931+0000","last_unstale":"2024-09-14T07:01:48.489545+0000","last_undegraded":"2024-09-14T07:01:48.489545+0000","last_fullsized":"2024-09-14T07:01:48.489545+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:34.355169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020075470000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1776,"num_read_kb":3899,"num_write":305,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"131'789","reported_seq":5674,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489195+0000","last_change":"2024-09-14T07:01:43.465640+0000","last_active":"2024-09-14T07:01:48.489195+0000","last_peered":"2024-09-14T07:01:48.489195+0000","last_clean":"2024-09-14T07:01:48.489195+0000","last_became_active":"2024-09-14T07:01:43.461464+0000","last_became_peered":"2024-09-14T07:01:43.461464+0000","last_unstale":"2024-09-14T07:01:48.489195+0000","last_undegraded":"2024-09-14T07:01:48.489195+0000","last_fullsized":"2024-09-14T07:01:48.489195+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:07.293229+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0087910980000000007,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6730,"num_read_kb":12616,"num_write":1057,"num_write_kb":7169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489294+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:01:48.489294+0000","last_peered":"2024-09-14T07:01:48.489294+0000","last_clean":"2024-09-14T07:01:48.489294+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:01:48.489294+0000","last_undegraded":"2024-09-14T07:01:48.489294+0000","last_fullsized":"2024-09-14T07:01:48.489294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"133'800","reported_seq":7023,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116939+0000","last_change":"2024-09-14T07:01:29.300111+0000","last_active":"2024-09-14T07:01:49.116939+0000","last_peered":"2024-09-14T07:01:49.116939+0000","last_clean":"2024-09-14T07:01:49.116939+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:01:49.116939+0000","last_undegraded":"2024-09-14T07:01:49.116939+0000","last_fullsized":"2024-09-14T07:01:49.116939+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:46.258973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010359880000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8046,"num_read_kb":9156,"num_write":887,"num_write_kb":5801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.117092+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:01:49.117092+0000","last_peered":"2024-09-14T07:01:49.117092+0000","last_clean":"2024-09-14T07:01:49.117092+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:01:49.117092+0000","last_undegraded":"2024-09-14T07:01:49.117092+0000","last_fullsized":"2024-09-14T07:01:49.117092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"132'510","reported_seq":11748,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116991+0000","last_change":"2024-09-14T07:01:36.306671+0000","last_active":"2024-09-14T07:01:49.116991+0000","last_peered":"2024-09-14T07:01:49.116991+0000","last_clean":"2024-09-14T07:01:49.116991+0000","last_became_active":"2024-09-14T07:01:36.305222+0000","last_became_peered":"2024-09-14T07:01:36.305222+0000","last_unstale":"2024-09-14T07:01:49.116991+0000","last_undegraded":"2024-09-14T07:01:49.116991+0000","last_fullsized":"2024-09-14T07:01:49.116991+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:12.004745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001033576,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12680,"num_read_kb":20099,"num_write":483,"num_write_kb":7950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489378+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:01:48.489378+0000","last_peered":"2024-09-14T07:01:48.489378+0000","last_clean":"2024-09-14T07:01:48.489378+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:01:48.489378+0000","last_undegraded":"2024-09-14T07:01:48.489378+0000","last_fullsized":"2024-09-14T07:01:48.489378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"129'508","reported_seq":3947,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237958+0000","last_change":"2024-09-14T07:01:47.492455+0000","last_active":"2024-09-14T07:01:49.237958+0000","last_peered":"2024-09-14T07:01:49.237958+0000","last_clean":"2024-09-14T07:01:49.237958+0000","last_became_active":"2024-09-14T07:01:47.490786+0000","last_became_peered":"2024-09-14T07:01:47.490786+0000","last_unstale":"2024-09-14T07:01:49.237958+0000","last_undegraded":"2024-09-14T07:01:49.237958+0000","last_fullsized":"2024-09-14T07:01:49.237958+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:11.019913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070556600000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4047,"num_read_kb":4338,"num_write":428,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":305,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489570+0000","last_change":"2024-09-14T07:01:33.416728+0000","last_active":"2024-09-14T07:01:48.489570+0000","last_peered":"2024-09-14T07:01:48.489570+0000","last_clean":"2024-09-14T07:01:48.489570+0000","last_became_active":"2024-09-14T07:01:33.416397+0000","last_became_peered":"2024-09-14T07:01:33.416397+0000","last_unstale":"2024-09-14T07:01:48.489570+0000","last_undegraded":"2024-09-14T07:01:48.489570+0000","last_fullsized":"2024-09-14T07:01:48.489570+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:56.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"132'573","reported_seq":10464,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116353+0000","last_change":"2024-09-14T07:01:36.304148+0000","last_active":"2024-09-14T07:01:49.116353+0000","last_peered":"2024-09-14T07:01:49.116353+0000","last_clean":"2024-09-14T07:01:49.116353+0000","last_became_active":"2024-09-14T07:01:36.302844+0000","last_became_peered":"2024-09-14T07:01:36.302844+0000","last_unstale":"2024-09-14T07:01:49.116353+0000","last_undegraded":"2024-09-14T07:01:49.116353+0000","last_fullsized":"2024-09-14T07:01:49.116353+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:45:52.752571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093736199999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8151,"num_read_kb":12914,"num_write":603,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"133'814","reported_seq":8542,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489516+0000","last_change":"2024-09-14T07:01:35.302802+0000","last_active":"2024-09-14T07:01:48.489516+0000","last_peered":"2024-09-14T07:01:48.489516+0000","last_clean":"2024-09-14T07:01:48.489516+0000","last_became_active":"2024-09-14T07:01:35.301421+0000","last_became_peered":"2024-09-14T07:01:35.301421+0000","last_unstale":"2024-09-14T07:01:48.489516+0000","last_undegraded":"2024-09-14T07:01:48.489516+0000","last_fullsized":"2024-09-14T07:01:48.489516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:41:57.632029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027805429999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":8837,"num_read_kb":11133,"num_write":1016,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":31,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489044+0000","last_change":"2024-09-14T07:01:40.447790+0000","last_active":"2024-09-14T07:01:48.489044+0000","last_peered":"2024-09-14T07:01:48.489044+0000","last_clean":"2024-09-14T07:01:48.489044+0000","last_became_active":"2024-09-14T07:01:40.444161+0000","last_became_peered":"2024-09-14T07:01:40.444161+0000","last_unstale":"2024-09-14T07:01:48.489044+0000","last_undegraded":"2024-09-14T07:01:48.489044+0000","last_fullsized":"2024-09-14T07:01:48.489044+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:14.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"131'477","reported_seq":22200,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489342+0000","last_change":"2024-09-14T07:01:29.368340+0000","last_active":"2024-09-14T07:01:48.489342+0000","last_peered":"2024-09-14T07:01:48.489342+0000","last_clean":"2024-09-14T07:01:48.489342+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:01:48.489342+0000","last_undegraded":"2024-09-14T07:01:48.489342+0000","last_fullsized":"2024-09-14T07:01:48.489342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:44.687681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089711299999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":22342,"num_read_kb":44381,"num_write":481,"num_write_kb":17365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":280,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116984+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:01:49.116984+0000","last_peered":"2024-09-14T07:01:49.116984+0000","last_clean":"2024-09-14T07:01:49.116984+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:01:49.116984+0000","last_undegraded":"2024-09-14T07:01:49.116984+0000","last_fullsized":"2024-09-14T07:01:49.116984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"129'389","reported_seq":2578,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237880+0000","last_change":"2024-09-14T07:01:47.487342+0000","last_active":"2024-09-14T07:01:49.237880+0000","last_peered":"2024-09-14T07:01:49.237880+0000","last_clean":"2024-09-14T07:01:49.237880+0000","last_became_active":"2024-09-14T07:01:47.485661+0000","last_became_peered":"2024-09-14T07:01:47.485661+0000","last_unstale":"2024-09-14T07:01:49.237880+0000","last_undegraded":"2024-09-14T07:01:49.237880+0000","last_fullsized":"2024-09-14T07:01:49.237880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:10.350553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602144,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3106,"num_read_kb":12645,"num_write":311,"num_write_kb":7126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488945+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T07:01:48.488945+0000","last_peered":"2024-09-14T07:01:48.488945+0000","last_clean":"2024-09-14T07:01:48.488945+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:01:48.488945+0000","last_undegraded":"2024-09-14T07:01:48.488945+0000","last_fullsized":"2024-09-14T07:01:48.488945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"132'710","reported_seq":10681,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.299674+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:09.671718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074696400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11440,"num_read_kb":36601,"num_write":924,"num_write_kb":12561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":302,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489125+0000","last_change":"2024-09-14T07:00:27.523385+0000","last_active":"2024-09-14T07:01:48.489125+0000","last_peered":"2024-09-14T07:01:48.489125+0000","last_clean":"2024-09-14T07:01:48.489125+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:01:48.489125+0000","last_undegraded":"2024-09-14T07:01:48.489125+0000","last_fullsized":"2024-09-14T07:01:48.489125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:19.504174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":371,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116500+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T07:01:49.116500+0000","last_peered":"2024-09-14T07:01:49.116500+0000","last_clean":"2024-09-14T07:01:49.116500+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:01:49.116500+0000","last_undegraded":"2024-09-14T07:01:49.116500+0000","last_fullsized":"2024-09-14T07:01:49.116500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"130'594","reported_seq":10088,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116783+0000","last_change":"2024-09-14T07:01:29.297332+0000","last_active":"2024-09-14T07:01:49.116783+0000","last_peered":"2024-09-14T07:01:49.116783+0000","last_clean":"2024-09-14T07:01:49.116783+0000","last_became_active":"2024-09-14T07:00:53.610102+0000","last_became_peered":"2024-09-14T07:00:53.610102+0000","last_unstale":"2024-09-14T07:01:49.116783+0000","last_undegraded":"2024-09-14T07:01:49.116783+0000","last_fullsized":"2024-09-14T07:01:49.116783+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:58.662454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097674599999999991,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":9511,"num_read_kb":20954,"num_write":534,"num_write_kb":10486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4712244,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"133'769","reported_seq":6051,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116875+0000","last_change":"2024-09-14T07:01:37.305171+0000","last_active":"2024-09-14T07:01:49.116875+0000","last_peered":"2024-09-14T07:01:49.116875+0000","last_clean":"2024-09-14T07:01:49.116875+0000","last_became_active":"2024-09-14T07:01:37.303369+0000","last_became_peered":"2024-09-14T07:01:37.303369+0000","last_unstale":"2024-09-14T07:01:49.116875+0000","last_undegraded":"2024-09-14T07:01:49.116875+0000","last_fullsized":"2024-09-14T07:01:49.116875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:52.195099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024576519999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8449,"num_read_kb":18168,"num_write":805,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"132'1123","reported_seq":8414,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489096+0000","last_change":"2024-09-14T07:01:29.295466+0000","last_active":"2024-09-14T07:01:48.489096+0000","last_peered":"2024-09-14T07:01:48.489096+0000","last_clean":"2024-09-14T07:01:48.489096+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:01:48.489096+0000","last_undegraded":"2024-09-14T07:01:48.489096+0000","last_fullsized":"2024-09-14T07:01:48.489096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:23:01.075768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00157712,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":10507,"num_read_kb":9740,"num_write":1539,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'352","reported_seq":3167,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489439+0000","last_change":"2024-09-14T07:01:36.307001+0000","last_active":"2024-09-14T07:01:48.489439+0000","last_peered":"2024-09-14T07:01:48.489439+0000","last_clean":"2024-09-14T07:01:48.489439+0000","last_became_active":"2024-09-14T07:01:36.305298+0000","last_became_peered":"2024-09-14T07:01:36.305298+0000","last_unstale":"2024-09-14T07:01:48.489439+0000","last_undegraded":"2024-09-14T07:01:48.489439+0000","last_fullsized":"2024-09-14T07:01:48.489439+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:38:17.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074805599999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1936,"num_read_kb":8361,"num_write":285,"num_write_kb":3165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"128'460","reported_seq":5812,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489731+0000","last_change":"2024-09-14T07:01:35.303666+0000","last_active":"2024-09-14T07:01:48.489731+0000","last_peered":"2024-09-14T07:01:48.489731+0000","last_clean":"2024-09-14T07:01:48.489731+0000","last_became_active":"2024-09-14T07:01:35.302427+0000","last_became_peered":"2024-09-14T07:01:35.302427+0000","last_unstale":"2024-09-14T07:01:48.489731+0000","last_undegraded":"2024-09-14T07:01:48.489731+0000","last_fullsized":"2024-09-14T07:01:48.489731+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:57.816084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724418,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5156,"num_read_kb":11268,"num_write":417,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"133'441","reported_seq":2271,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489369+0000","last_change":"2024-09-14T07:01:29.295502+0000","last_active":"2024-09-14T07:01:48.489369+0000","last_peered":"2024-09-14T07:01:48.489369+0000","last_clean":"2024-09-14T07:01:48.489369+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:01:48.489369+0000","last_undegraded":"2024-09-14T07:01:48.489369+0000","last_fullsized":"2024-09-14T07:01:48.489369+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:12.777971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015159500000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1920,"num_read_kb":6604,"num_write":451,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"132'450","reported_seq":12399,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116494+0000","last_change":"2024-09-14T07:01:29.344029+0000","last_active":"2024-09-14T07:01:49.116494+0000","last_peered":"2024-09-14T07:01:49.116494+0000","last_clean":"2024-09-14T07:01:49.116494+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:01:49.116494+0000","last_undegraded":"2024-09-14T07:01:49.116494+0000","last_fullsized":"2024-09-14T07:01:49.116494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:57.081265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728802,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":12086,"num_read_kb":54262,"num_write":423,"num_write_kb":18409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"130'507","reported_seq":4859,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489784+0000","last_change":"2024-09-14T07:01:29.346684+0000","last_active":"2024-09-14T07:01:48.489784+0000","last_peered":"2024-09-14T07:01:48.489784+0000","last_clean":"2024-09-14T07:01:48.489784+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:01:48.489784+0000","last_undegraded":"2024-09-14T07:01:48.489784+0000","last_fullsized":"2024-09-14T07:01:48.489784+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:59:29.173513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001321022,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5747,"num_read_kb":11430,"num_write":424,"num_write_kb":4665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"131'440","reported_seq":4097,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116394+0000","last_change":"2024-09-14T07:01:29.295627+0000","last_active":"2024-09-14T07:01:49.116394+0000","last_peered":"2024-09-14T07:01:49.116394+0000","last_clean":"2024-09-14T07:01:49.116394+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:01:49.116394+0000","last_undegraded":"2024-09-14T07:01:49.116394+0000","last_fullsized":"2024-09-14T07:01:49.116394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:58.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095644100000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3634,"num_read_kb":12358,"num_write":404,"num_write_kb":6155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"130'622","reported_seq":11103,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.238002+0000","last_change":"2024-09-14T07:01:47.492336+0000","last_active":"2024-09-14T07:01:49.238002+0000","last_peered":"2024-09-14T07:01:49.238002+0000","last_clean":"2024-09-14T07:01:49.238002+0000","last_became_active":"2024-09-14T07:01:47.490646+0000","last_became_peered":"2024-09-14T07:01:47.490646+0000","last_unstale":"2024-09-14T07:01:49.238002+0000","last_undegraded":"2024-09-14T07:01:49.238002+0000","last_fullsized":"2024-09-14T07:01:49.238002+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:06:20.079036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001614826,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12269,"num_read_kb":24191,"num_write":663,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"131'725","reported_seq":7151,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237968+0000","last_change":"2024-09-14T07:01:47.487409+0000","last_active":"2024-09-14T07:01:49.237968+0000","last_peered":"2024-09-14T07:01:49.237968+0000","last_clean":"2024-09-14T07:01:49.237968+0000","last_became_active":"2024-09-14T07:01:47.485532+0000","last_became_peered":"2024-09-14T07:01:47.485532+0000","last_unstale":"2024-09-14T07:01:49.237968+0000","last_undegraded":"2024-09-14T07:01:49.237968+0000","last_fullsized":"2024-09-14T07:01:49.237968+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:55.605522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071147799999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8637,"num_read_kb":13164,"num_write":610,"num_write_kb":4987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"131'615","reported_seq":6529,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488972+0000","last_change":"2024-09-14T07:01:29.296192+0000","last_active":"2024-09-14T07:01:48.488972+0000","last_peered":"2024-09-14T07:01:48.488972+0000","last_clean":"2024-09-14T07:01:48.488972+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:01:48.488972+0000","last_undegraded":"2024-09-14T07:01:48.488972+0000","last_fullsized":"2024-09-14T07:01:48.488972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:49:30.451851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019616249999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6772,"num_read_kb":10244,"num_write":478,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":579,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":575,"num_read":241540,"num_read_kb":530952,"num_write":22797,"num_write_kb":271683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3188,"num_bytes_recovered":84516668,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20427,"ondisk_log_size":20427,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41276,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330564,"statfs":{"total":96636764160,"available":96594497536,"internally_reserved":0,"allocated":1810432,"data_stored":872232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59344,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312496,"statfs":{"total":96636764160,"available":96575995904,"internally_reserved":0,"allocated":1822720,"data_stored":878219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59332,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312508,"statfs":{"total":96636764160,"available":96576008192,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64948,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306892,"statfs":{"total":96636764160,"available":96570257408,"internally_reserved":0,"allocated":1867776,"data_stored":1462600,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69072,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302768,"statfs":{"total":96636764160,"available":96566034432,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1880064,"data_stored":1468587,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45436,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326404,"statfs":{"total":96636764160,"available":96590237696,"internally_reserved":0,"allocated":1806336,"data_stored":872213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:01:52.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:01:52.421 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:52.421 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:01:52.433 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-14T07:01:51.777824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":575,"num_read":241606,"num_read_kb":531007,"num_write":22900,"num_write_kb":273313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3193,"num_bytes_recovered":84516725,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20502,"ondisk_log_size":20502,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":431780,"kb_used_data":14300,"kb_used_omap":12,"kb_used_meta":417267,"kb_avail":754542940,"statfs":{"total":773094113280,"available":772651970560,"internally_reserved":0,"allocated":14643200,"data_stored":8188451,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":427281969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001513"},"pg_stats":[{"pgid":"3.1f","version":"133'538","reported_seq":7941,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489581+0000","last_change":"2024-09-14T07:01:29.346010+0000","last_active":"2024-09-14T07:01:48.489581+0000","last_peered":"2024-09-14T07:01:48.489581+0000","last_clean":"2024-09-14T07:01:48.489581+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:01:48.489581+0000","last_undegraded":"2024-09-14T07:01:48.489581+0000","last_fullsized":"2024-09-14T07:01:48.489581+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:00.019940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099215999999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8424,"num_read_kb":37327,"num_write":508,"num_write_kb":21659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"130'423","reported_seq":3021,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489455+0000","last_change":"2024-09-14T07:01:29.367999+0000","last_active":"2024-09-14T07:01:48.489455+0000","last_peered":"2024-09-14T07:01:48.489455+0000","last_clean":"2024-09-14T07:01:48.489455+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:01:48.489455+0000","last_undegraded":"2024-09-14T07:01:48.489455+0000","last_fullsized":"2024-09-14T07:01:48.489455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:38.115466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001057501,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2716,"num_read_kb":8402,"num_write":360,"num_write_kb":7998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"130'339","reported_seq":5823,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:43.465257+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T07:01:43.462852+0000","last_became_peered":"2024-09-14T07:01:43.462852+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:36.409964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000797209,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5182,"num_read_kb":12861,"num_write":295,"num_write_kb":5166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"133'2042","reported_seq":4848,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116619+0000","last_change":"2024-09-14T07:01:29.296724+0000","last_active":"2024-09-14T07:01:49.116619+0000","last_peered":"2024-09-14T07:01:49.116619+0000","last_clean":"2024-09-14T07:01:49.116619+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:01:49.116619+0000","last_undegraded":"2024-09-14T07:01:49.116619+0000","last_fullsized":"2024-09-14T07:01:49.116619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":2042,"log_dups_size":0,"ondisk_log_size":2042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:44.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096933700000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11671,"num_read_kb":22543,"num_write":3453,"num_write_kb":11219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"131'900","reported_seq":3379,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116967+0000","last_change":"2024-09-14T07:01:29.299798+0000","last_active":"2024-09-14T07:01:49.116967+0000","last_peered":"2024-09-14T07:01:49.116967+0000","last_clean":"2024-09-14T07:01:49.116967+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:01:49.116967+0000","last_undegraded":"2024-09-14T07:01:49.116967+0000","last_fullsized":"2024-09-14T07:01:49.116967+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:51.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087443700000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4436,"num_read_kb":12231,"num_write":1137,"num_write_kb":20066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"133'1187","reported_seq":8951,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116584+0000","last_change":"2024-09-14T07:01:29.296969+0000","last_active":"2024-09-14T07:01:49.116584+0000","last_peered":"2024-09-14T07:01:49.116584+0000","last_clean":"2024-09-14T07:01:49.116584+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:01:49.116584+0000","last_undegraded":"2024-09-14T07:01:49.116584+0000","last_fullsized":"2024-09-14T07:01:49.116584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:26:48.784038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015092009999999999,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":13058,"num_read_kb":21088,"num_write":1639,"num_write_kb":6895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"133'517","reported_seq":5015,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.300460+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:54:21.579346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00137473,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4742,"num_read_kb":12493,"num_write":520,"num_write_kb":11830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"132'662","reported_seq":3807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:29.346100+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:15.330383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011507799999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5024,"num_read_kb":10270,"num_write":682,"num_write_kb":6392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"130'392","reported_seq":6847,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489521+0000","last_change":"2024-09-14T07:01:29.368091+0000","last_active":"2024-09-14T07:01:48.489521+0000","last_peered":"2024-09-14T07:01:48.489521+0000","last_clean":"2024-09-14T07:01:48.489521+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:01:48.489521+0000","last_undegraded":"2024-09-14T07:01:48.489521+0000","last_fullsized":"2024-09-14T07:01:48.489521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:17.259452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010952469999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7606,"num_read_kb":16770,"num_write":322,"num_write_kb":2846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'413","reported_seq":5194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116357+0000","last_change":"2024-09-14T07:01:43.464245+0000","last_active":"2024-09-14T07:01:49.116357+0000","last_peered":"2024-09-14T07:01:49.116357+0000","last_clean":"2024-09-14T07:01:49.116357+0000","last_became_active":"2024-09-14T07:01:43.462372+0000","last_became_peered":"2024-09-14T07:01:43.462372+0000","last_unstale":"2024-09-14T07:01:49.116357+0000","last_undegraded":"2024-09-14T07:01:49.116357+0000","last_fullsized":"2024-09-14T07:01:49.116357+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:32:17.252003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022563269999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4902,"num_read_kb":8441,"num_write":353,"num_write_kb":3835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"130'346","reported_seq":2783,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489545+0000","last_change":"2024-09-14T07:01:36.306565+0000","last_active":"2024-09-14T07:01:48.489545+0000","last_peered":"2024-09-14T07:01:48.489545+0000","last_clean":"2024-09-14T07:01:48.489545+0000","last_became_active":"2024-09-14T07:01:36.304931+0000","last_became_peered":"2024-09-14T07:01:36.304931+0000","last_unstale":"2024-09-14T07:01:48.489545+0000","last_undegraded":"2024-09-14T07:01:48.489545+0000","last_fullsized":"2024-09-14T07:01:48.489545+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:34.355169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020075470000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1776,"num_read_kb":3899,"num_write":305,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"131'789","reported_seq":5674,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489195+0000","last_change":"2024-09-14T07:01:43.465640+0000","last_active":"2024-09-14T07:01:48.489195+0000","last_peered":"2024-09-14T07:01:48.489195+0000","last_clean":"2024-09-14T07:01:48.489195+0000","last_became_active":"2024-09-14T07:01:43.461464+0000","last_became_peered":"2024-09-14T07:01:43.461464+0000","last_unstale":"2024-09-14T07:01:48.489195+0000","last_undegraded":"2024-09-14T07:01:48.489195+0000","last_fullsized":"2024-09-14T07:01:48.489195+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:07.293229+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0087910980000000007,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6730,"num_read_kb":12616,"num_write":1057,"num_write_kb":7169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489294+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:01:48.489294+0000","last_peered":"2024-09-14T07:01:48.489294+0000","last_clean":"2024-09-14T07:01:48.489294+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:01:48.489294+0000","last_undegraded":"2024-09-14T07:01:48.489294+0000","last_fullsized":"2024-09-14T07:01:48.489294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"133'800","reported_seq":7023,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116939+0000","last_change":"2024-09-14T07:01:29.300111+0000","last_active":"2024-09-14T07:01:49.116939+0000","last_peered":"2024-09-14T07:01:49.116939+0000","last_clean":"2024-09-14T07:01:49.116939+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:01:49.116939+0000","last_undegraded":"2024-09-14T07:01:49.116939+0000","last_fullsized":"2024-09-14T07:01:49.116939+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:46.258973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010359880000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8046,"num_read_kb":9156,"num_write":887,"num_write_kb":5801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.117092+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:01:49.117092+0000","last_peered":"2024-09-14T07:01:49.117092+0000","last_clean":"2024-09-14T07:01:49.117092+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:01:49.117092+0000","last_undegraded":"2024-09-14T07:01:49.117092+0000","last_fullsized":"2024-09-14T07:01:49.117092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"132'510","reported_seq":11748,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116991+0000","last_change":"2024-09-14T07:01:36.306671+0000","last_active":"2024-09-14T07:01:49.116991+0000","last_peered":"2024-09-14T07:01:49.116991+0000","last_clean":"2024-09-14T07:01:49.116991+0000","last_became_active":"2024-09-14T07:01:36.305222+0000","last_became_peered":"2024-09-14T07:01:36.305222+0000","last_unstale":"2024-09-14T07:01:49.116991+0000","last_undegraded":"2024-09-14T07:01:49.116991+0000","last_fullsized":"2024-09-14T07:01:49.116991+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:12.004745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001033576,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12680,"num_read_kb":20099,"num_write":483,"num_write_kb":7950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489378+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:01:48.489378+0000","last_peered":"2024-09-14T07:01:48.489378+0000","last_clean":"2024-09-14T07:01:48.489378+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:01:48.489378+0000","last_undegraded":"2024-09-14T07:01:48.489378+0000","last_fullsized":"2024-09-14T07:01:48.489378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"129'508","reported_seq":3947,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237958+0000","last_change":"2024-09-14T07:01:47.492455+0000","last_active":"2024-09-14T07:01:49.237958+0000","last_peered":"2024-09-14T07:01:49.237958+0000","last_clean":"2024-09-14T07:01:49.237958+0000","last_became_active":"2024-09-14T07:01:47.490786+0000","last_became_peered":"2024-09-14T07:01:47.490786+0000","last_unstale":"2024-09-14T07:01:49.237958+0000","last_undegraded":"2024-09-14T07:01:49.237958+0000","last_fullsized":"2024-09-14T07:01:49.237958+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:11.019913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070556600000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4047,"num_read_kb":4338,"num_write":428,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":305,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489570+0000","last_change":"2024-09-14T07:01:33.416728+0000","last_active":"2024-09-14T07:01:48.489570+0000","last_peered":"2024-09-14T07:01:48.489570+0000","last_clean":"2024-09-14T07:01:48.489570+0000","last_became_active":"2024-09-14T07:01:33.416397+0000","last_became_peered":"2024-09-14T07:01:33.416397+0000","last_unstale":"2024-09-14T07:01:48.489570+0000","last_undegraded":"2024-09-14T07:01:48.489570+0000","last_fullsized":"2024-09-14T07:01:48.489570+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:56.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"132'573","reported_seq":10464,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116353+0000","last_change":"2024-09-14T07:01:36.304148+0000","last_active":"2024-09-14T07:01:49.116353+0000","last_peered":"2024-09-14T07:01:49.116353+0000","last_clean":"2024-09-14T07:01:49.116353+0000","last_became_active":"2024-09-14T07:01:36.302844+0000","last_became_peered":"2024-09-14T07:01:36.302844+0000","last_unstale":"2024-09-14T07:01:49.116353+0000","last_undegraded":"2024-09-14T07:01:49.116353+0000","last_fullsized":"2024-09-14T07:01:49.116353+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:45:52.752571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093736199999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8151,"num_read_kb":12914,"num_write":603,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"133'814","reported_seq":8542,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489516+0000","last_change":"2024-09-14T07:01:35.302802+0000","last_active":"2024-09-14T07:01:48.489516+0000","last_peered":"2024-09-14T07:01:48.489516+0000","last_clean":"2024-09-14T07:01:48.489516+0000","last_became_active":"2024-09-14T07:01:35.301421+0000","last_became_peered":"2024-09-14T07:01:35.301421+0000","last_unstale":"2024-09-14T07:01:48.489516+0000","last_undegraded":"2024-09-14T07:01:48.489516+0000","last_fullsized":"2024-09-14T07:01:48.489516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:41:57.632029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027805429999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":8837,"num_read_kb":11133,"num_write":1016,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":31,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489044+0000","last_change":"2024-09-14T07:01:40.447790+0000","last_active":"2024-09-14T07:01:48.489044+0000","last_peered":"2024-09-14T07:01:48.489044+0000","last_clean":"2024-09-14T07:01:48.489044+0000","last_became_active":"2024-09-14T07:01:40.444161+0000","last_became_peered":"2024-09-14T07:01:40.444161+0000","last_unstale":"2024-09-14T07:01:48.489044+0000","last_undegraded":"2024-09-14T07:01:48.489044+0000","last_fullsized":"2024-09-14T07:01:48.489044+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:14.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"131'477","reported_seq":22200,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489342+0000","last_change":"2024-09-14T07:01:29.368340+0000","last_active":"2024-09-14T07:01:48.489342+0000","last_peered":"2024-09-14T07:01:48.489342+0000","last_clean":"2024-09-14T07:01:48.489342+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:01:48.489342+0000","last_undegraded":"2024-09-14T07:01:48.489342+0000","last_fullsized":"2024-09-14T07:01:48.489342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:44.687681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089711299999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":22342,"num_read_kb":44381,"num_write":481,"num_write_kb":17365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":280,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116984+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:01:49.116984+0000","last_peered":"2024-09-14T07:01:49.116984+0000","last_clean":"2024-09-14T07:01:49.116984+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:01:49.116984+0000","last_undegraded":"2024-09-14T07:01:49.116984+0000","last_fullsized":"2024-09-14T07:01:49.116984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"129'389","reported_seq":2578,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237880+0000","last_change":"2024-09-14T07:01:47.487342+0000","last_active":"2024-09-14T07:01:49.237880+0000","last_peered":"2024-09-14T07:01:49.237880+0000","last_clean":"2024-09-14T07:01:49.237880+0000","last_became_active":"2024-09-14T07:01:47.485661+0000","last_became_peered":"2024-09-14T07:01:47.485661+0000","last_unstale":"2024-09-14T07:01:49.237880+0000","last_undegraded":"2024-09-14T07:01:49.237880+0000","last_fullsized":"2024-09-14T07:01:49.237880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:10.350553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602144,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3106,"num_read_kb":12645,"num_write":311,"num_write_kb":7126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488945+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T07:01:48.488945+0000","last_peered":"2024-09-14T07:01:48.488945+0000","last_clean":"2024-09-14T07:01:48.488945+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:01:48.488945+0000","last_undegraded":"2024-09-14T07:01:48.488945+0000","last_fullsized":"2024-09-14T07:01:48.488945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"132'710","reported_seq":10681,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.299674+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:09.671718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074696400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11440,"num_read_kb":36601,"num_write":924,"num_write_kb":12561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":302,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489125+0000","last_change":"2024-09-14T07:00:27.523385+0000","last_active":"2024-09-14T07:01:48.489125+0000","last_peered":"2024-09-14T07:01:48.489125+0000","last_clean":"2024-09-14T07:01:48.489125+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:01:48.489125+0000","last_undegraded":"2024-09-14T07:01:48.489125+0000","last_fullsized":"2024-09-14T07:01:48.489125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:19.504174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":371,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116500+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T07:01:49.116500+0000","last_peered":"2024-09-14T07:01:49.116500+0000","last_clean":"2024-09-14T07:01:49.116500+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:01:49.116500+0000","last_undegraded":"2024-09-14T07:01:49.116500+0000","last_fullsized":"2024-09-14T07:01:49.116500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"130'594","reported_seq":10088,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116783+0000","last_change":"2024-09-14T07:01:29.297332+0000","last_active":"2024-09-14T07:01:49.116783+0000","last_peered":"2024-09-14T07:01:49.116783+0000","last_clean":"2024-09-14T07:01:49.116783+0000","last_became_active":"2024-09-14T07:00:53.610102+0000","last_became_peered":"2024-09-14T07:00:53.610102+0000","last_unstale":"2024-09-14T07:01:49.116783+0000","last_undegraded":"2024-09-14T07:01:49.116783+0000","last_fullsized":"2024-09-14T07:01:49.116783+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:58.662454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097674599999999991,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":9511,"num_read_kb":20954,"num_write":534,"num_write_kb":10486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4712244,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"133'769","reported_seq":6051,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116875+0000","last_change":"2024-09-14T07:01:37.305171+0000","last_active":"2024-09-14T07:01:49.116875+0000","last_peered":"2024-09-14T07:01:49.116875+0000","last_clean":"2024-09-14T07:01:49.116875+0000","last_became_active":"2024-09-14T07:01:37.303369+0000","last_became_peered":"2024-09-14T07:01:37.303369+0000","last_unstale":"2024-09-14T07:01:49.116875+0000","last_undegraded":"2024-09-14T07:01:49.116875+0000","last_fullsized":"2024-09-14T07:01:49.116875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:52.195099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024576519999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8449,"num_read_kb":18168,"num_write":805,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"132'1123","reported_seq":8414,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489096+0000","last_change":"2024-09-14T07:01:29.295466+0000","last_active":"2024-09-14T07:01:48.489096+0000","last_peered":"2024-09-14T07:01:48.489096+0000","last_clean":"2024-09-14T07:01:48.489096+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:01:48.489096+0000","last_undegraded":"2024-09-14T07:01:48.489096+0000","last_fullsized":"2024-09-14T07:01:48.489096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:23:01.075768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00157712,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":10507,"num_read_kb":9740,"num_write":1539,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'352","reported_seq":3167,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489439+0000","last_change":"2024-09-14T07:01:36.307001+0000","last_active":"2024-09-14T07:01:48.489439+0000","last_peered":"2024-09-14T07:01:48.489439+0000","last_clean":"2024-09-14T07:01:48.489439+0000","last_became_active":"2024-09-14T07:01:36.305298+0000","last_became_peered":"2024-09-14T07:01:36.305298+0000","last_unstale":"2024-09-14T07:01:48.489439+0000","last_undegraded":"2024-09-14T07:01:48.489439+0000","last_fullsized":"2024-09-14T07:01:48.489439+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:38:17.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074805599999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1936,"num_read_kb":8361,"num_write":285,"num_write_kb":3165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"128'460","reported_seq":5812,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489731+0000","last_change":"2024-09-14T07:01:35.303666+0000","last_active":"2024-09-14T07:01:48.489731+0000","last_peered":"2024-09-14T07:01:48.489731+0000","last_clean":"2024-09-14T07:01:48.489731+0000","last_became_active":"2024-09-14T07:01:35.302427+0000","last_became_peered":"2024-09-14T07:01:35.302427+0000","last_unstale":"2024-09-14T07:01:48.489731+0000","last_undegraded":"2024-09-14T07:01:48.489731+0000","last_fullsized":"2024-09-14T07:01:48.489731+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:57.816084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724418,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5156,"num_read_kb":11268,"num_write":417,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"133'441","reported_seq":2271,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489369+0000","last_change":"2024-09-14T07:01:29.295502+0000","last_active":"2024-09-14T07:01:48.489369+0000","last_peered":"2024-09-14T07:01:48.489369+0000","last_clean":"2024-09-14T07:01:48.489369+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:01:48.489369+0000","last_undegraded":"2024-09-14T07:01:48.489369+0000","last_fullsized":"2024-09-14T07:01:48.489369+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:12.777971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015159500000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1920,"num_read_kb":6604,"num_write":451,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"132'450","reported_seq":12399,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116494+0000","last_change":"2024-09-14T07:01:29.344029+0000","last_active":"2024-09-14T07:01:49.116494+0000","last_peered":"2024-09-14T07:01:49.116494+0000","last_clean":"2024-09-14T07:01:49.116494+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:01:49.116494+0000","last_undegraded":"2024-09-14T07:01:49.116494+0000","last_fullsized":"2024-09-14T07:01:49.116494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:57.081265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728802,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":12086,"num_read_kb":54262,"num_write":423,"num_write_kb":18409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"130'507","reported_seq":4859,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489784+0000","last_change":"2024-09-14T07:01:29.346684+0000","last_active":"2024-09-14T07:01:48.489784+0000","last_peered":"2024-09-14T07:01:48.489784+0000","last_clean":"2024-09-14T07:01:48.489784+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:01:48.489784+0000","last_undegraded":"2024-09-14T07:01:48.489784+0000","last_fullsized":"2024-09-14T07:01:48.489784+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:59:29.173513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001321022,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5747,"num_read_kb":11430,"num_write":424,"num_write_kb":4665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"131'440","reported_seq":4097,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116394+0000","last_change":"2024-09-14T07:01:29.295627+0000","last_active":"2024-09-14T07:01:49.116394+0000","last_peered":"2024-09-14T07:01:49.116394+0000","last_clean":"2024-09-14T07:01:49.116394+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:01:49.116394+0000","last_undegraded":"2024-09-14T07:01:49.116394+0000","last_fullsized":"2024-09-14T07:01:49.116394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:58.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095644100000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3634,"num_read_kb":12358,"num_write":404,"num_write_kb":6155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"130'622","reported_seq":11103,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.238002+0000","last_change":"2024-09-14T07:01:47.492336+0000","last_active":"2024-09-14T07:01:49.238002+0000","last_peered":"2024-09-14T07:01:49.238002+0000","last_clean":"2024-09-14T07:01:49.238002+0000","last_became_active":"2024-09-14T07:01:47.490646+0000","last_became_peered":"2024-09-14T07:01:47.490646+0000","last_unstale":"2024-09-14T07:01:49.238002+0000","last_undegraded":"2024-09-14T07:01:49.238002+0000","last_fullsized":"2024-09-14T07:01:49.238002+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:06:20.079036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001614826,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12269,"num_read_kb":24191,"num_write":663,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"131'725","reported_seq":7151,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237968+0000","last_change":"2024-09-14T07:01:47.487409+0000","last_active":"2024-09-14T07:01:49.237968+0000","last_peered":"2024-09-14T07:01:49.237968+0000","last_clean":"2024-09-14T07:01:49.237968+0000","last_became_active":"2024-09-14T07:01:47.485532+0000","last_became_peered":"2024-09-14T07:01:47.485532+0000","last_unstale":"2024-09-14T07:01:49.237968+0000","last_undegraded":"2024-09-14T07:01:49.237968+0000","last_fullsized":"2024-09-14T07:01:49.237968+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:55.605522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071147799999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8637,"num_read_kb":13164,"num_write":610,"num_write_kb":4987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"131'615","reported_seq":6529,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488972+0000","last_change":"2024-09-14T07:01:29.296192+0000","last_active":"2024-09-14T07:01:48.488972+0000","last_peered":"2024-09-14T07:01:48.488972+0000","last_clean":"2024-09-14T07:01:48.488972+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:01:48.488972+0000","last_undegraded":"2024-09-14T07:01:48.488972+0000","last_fullsized":"2024-09-14T07:01:48.488972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:49:30.451851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019616249999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6772,"num_read_kb":10244,"num_write":478,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":579,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":575,"num_read":241540,"num_read_kb":530952,"num_write":22797,"num_write_kb":271683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3188,"num_bytes_recovered":84516668,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20427,"ondisk_log_size":20427,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41276,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330564,"statfs":{"total":96636764160,"available":96594497536,"internally_reserved":0,"allocated":1810432,"data_stored":872232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59344,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312496,"statfs":{"total":96636764160,"available":96575995904,"internally_reserved":0,"allocated":1822720,"data_stored":878219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59332,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312508,"statfs":{"total":96636764160,"available":96576008192,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64948,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306892,"statfs":{"total":96636764160,"available":96570257408,"internally_reserved":0,"allocated":1867776,"data_stored":1462600,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69072,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302768,"statfs":{"total":96636764160,"available":96566034432,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1880064,"data_stored":1468587,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45436,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326404,"statfs":{"total":96636764160,"available":96590237696,"internally_reserved":0,"allocated":1806336,"data_stored":872213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:01:52.435 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T07:01:52.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-14T07:01:52.750 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-14T07:01:52.760 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T07:01:52.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:01:53.058 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:53.059 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:01:53.071 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-14T07:01:51.777824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":575,"num_read":241606,"num_read_kb":531007,"num_write":22900,"num_write_kb":273313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3193,"num_bytes_recovered":84516725,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20502,"ondisk_log_size":20502,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":431780,"kb_used_data":14300,"kb_used_omap":12,"kb_used_meta":417267,"kb_avail":754542940,"statfs":{"total":773094113280,"available":772651970560,"internally_reserved":0,"allocated":14643200,"data_stored":8188451,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":427281969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001513"},"pg_stats":[{"pgid":"3.1f","version":"133'538","reported_seq":7941,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489581+0000","last_change":"2024-09-14T07:01:29.346010+0000","last_active":"2024-09-14T07:01:48.489581+0000","last_peered":"2024-09-14T07:01:48.489581+0000","last_clean":"2024-09-14T07:01:48.489581+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:01:48.489581+0000","last_undegraded":"2024-09-14T07:01:48.489581+0000","last_fullsized":"2024-09-14T07:01:48.489581+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:00.019940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099215999999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8424,"num_read_kb":37327,"num_write":508,"num_write_kb":21659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"130'423","reported_seq":3021,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489455+0000","last_change":"2024-09-14T07:01:29.367999+0000","last_active":"2024-09-14T07:01:48.489455+0000","last_peered":"2024-09-14T07:01:48.489455+0000","last_clean":"2024-09-14T07:01:48.489455+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:01:48.489455+0000","last_undegraded":"2024-09-14T07:01:48.489455+0000","last_fullsized":"2024-09-14T07:01:48.489455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:38.115466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001057501,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2716,"num_read_kb":8402,"num_write":360,"num_write_kb":7998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"130'339","reported_seq":5823,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:43.465257+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T07:01:43.462852+0000","last_became_peered":"2024-09-14T07:01:43.462852+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:36.409964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000797209,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5182,"num_read_kb":12861,"num_write":295,"num_write_kb":5166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"133'2042","reported_seq":4848,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116619+0000","last_change":"2024-09-14T07:01:29.296724+0000","last_active":"2024-09-14T07:01:49.116619+0000","last_peered":"2024-09-14T07:01:49.116619+0000","last_clean":"2024-09-14T07:01:49.116619+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:01:49.116619+0000","last_undegraded":"2024-09-14T07:01:49.116619+0000","last_fullsized":"2024-09-14T07:01:49.116619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":2042,"log_dups_size":0,"ondisk_log_size":2042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:44.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096933700000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11671,"num_read_kb":22543,"num_write":3453,"num_write_kb":11219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"131'900","reported_seq":3379,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116967+0000","last_change":"2024-09-14T07:01:29.299798+0000","last_active":"2024-09-14T07:01:49.116967+0000","last_peered":"2024-09-14T07:01:49.116967+0000","last_clean":"2024-09-14T07:01:49.116967+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:01:49.116967+0000","last_undegraded":"2024-09-14T07:01:49.116967+0000","last_fullsized":"2024-09-14T07:01:49.116967+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:51.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087443700000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4436,"num_read_kb":12231,"num_write":1137,"num_write_kb":20066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"133'1187","reported_seq":8951,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116584+0000","last_change":"2024-09-14T07:01:29.296969+0000","last_active":"2024-09-14T07:01:49.116584+0000","last_peered":"2024-09-14T07:01:49.116584+0000","last_clean":"2024-09-14T07:01:49.116584+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:01:49.116584+0000","last_undegraded":"2024-09-14T07:01:49.116584+0000","last_fullsized":"2024-09-14T07:01:49.116584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:26:48.784038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015092009999999999,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":13058,"num_read_kb":21088,"num_write":1639,"num_write_kb":6895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"133'517","reported_seq":5015,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.300460+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:54:21.579346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00137473,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4742,"num_read_kb":12493,"num_write":520,"num_write_kb":11830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"132'662","reported_seq":3807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:29.346100+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:15.330383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011507799999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5024,"num_read_kb":10270,"num_write":682,"num_write_kb":6392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"130'392","reported_seq":6847,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489521+0000","last_change":"2024-09-14T07:01:29.368091+0000","last_active":"2024-09-14T07:01:48.489521+0000","last_peered":"2024-09-14T07:01:48.489521+0000","last_clean":"2024-09-14T07:01:48.489521+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:01:48.489521+0000","last_undegraded":"2024-09-14T07:01:48.489521+0000","last_fullsized":"2024-09-14T07:01:48.489521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:17.259452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010952469999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7606,"num_read_kb":16770,"num_write":322,"num_write_kb":2846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'413","reported_seq":5194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116357+0000","last_change":"2024-09-14T07:01:43.464245+0000","last_active":"2024-09-14T07:01:49.116357+0000","last_peered":"2024-09-14T07:01:49.116357+0000","last_clean":"2024-09-14T07:01:49.116357+0000","last_became_active":"2024-09-14T07:01:43.462372+0000","last_became_peered":"2024-09-14T07:01:43.462372+0000","last_unstale":"2024-09-14T07:01:49.116357+0000","last_undegraded":"2024-09-14T07:01:49.116357+0000","last_fullsized":"2024-09-14T07:01:49.116357+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:32:17.252003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022563269999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4902,"num_read_kb":8441,"num_write":353,"num_write_kb":3835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"130'346","reported_seq":2783,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489545+0000","last_change":"2024-09-14T07:01:36.306565+0000","last_active":"2024-09-14T07:01:48.489545+0000","last_peered":"2024-09-14T07:01:48.489545+0000","last_clean":"2024-09-14T07:01:48.489545+0000","last_became_active":"2024-09-14T07:01:36.304931+0000","last_became_peered":"2024-09-14T07:01:36.304931+0000","last_unstale":"2024-09-14T07:01:48.489545+0000","last_undegraded":"2024-09-14T07:01:48.489545+0000","last_fullsized":"2024-09-14T07:01:48.489545+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:34.355169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020075470000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1776,"num_read_kb":3899,"num_write":305,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"131'789","reported_seq":5674,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489195+0000","last_change":"2024-09-14T07:01:43.465640+0000","last_active":"2024-09-14T07:01:48.489195+0000","last_peered":"2024-09-14T07:01:48.489195+0000","last_clean":"2024-09-14T07:01:48.489195+0000","last_became_active":"2024-09-14T07:01:43.461464+0000","last_became_peered":"2024-09-14T07:01:43.461464+0000","last_unstale":"2024-09-14T07:01:48.489195+0000","last_undegraded":"2024-09-14T07:01:48.489195+0000","last_fullsized":"2024-09-14T07:01:48.489195+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:07.293229+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0087910980000000007,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6730,"num_read_kb":12616,"num_write":1057,"num_write_kb":7169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489294+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:01:48.489294+0000","last_peered":"2024-09-14T07:01:48.489294+0000","last_clean":"2024-09-14T07:01:48.489294+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:01:48.489294+0000","last_undegraded":"2024-09-14T07:01:48.489294+0000","last_fullsized":"2024-09-14T07:01:48.489294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"133'800","reported_seq":7023,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116939+0000","last_change":"2024-09-14T07:01:29.300111+0000","last_active":"2024-09-14T07:01:49.116939+0000","last_peered":"2024-09-14T07:01:49.116939+0000","last_clean":"2024-09-14T07:01:49.116939+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:01:49.116939+0000","last_undegraded":"2024-09-14T07:01:49.116939+0000","last_fullsized":"2024-09-14T07:01:49.116939+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:46.258973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010359880000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8046,"num_read_kb":9156,"num_write":887,"num_write_kb":5801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.117092+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:01:49.117092+0000","last_peered":"2024-09-14T07:01:49.117092+0000","last_clean":"2024-09-14T07:01:49.117092+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:01:49.117092+0000","last_undegraded":"2024-09-14T07:01:49.117092+0000","last_fullsized":"2024-09-14T07:01:49.117092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"132'510","reported_seq":11748,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116991+0000","last_change":"2024-09-14T07:01:36.306671+0000","last_active":"2024-09-14T07:01:49.116991+0000","last_peered":"2024-09-14T07:01:49.116991+0000","last_clean":"2024-09-14T07:01:49.116991+0000","last_became_active":"2024-09-14T07:01:36.305222+0000","last_became_peered":"2024-09-14T07:01:36.305222+0000","last_unstale":"2024-09-14T07:01:49.116991+0000","last_undegraded":"2024-09-14T07:01:49.116991+0000","last_fullsized":"2024-09-14T07:01:49.116991+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:12.004745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001033576,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12680,"num_read_kb":20099,"num_write":483,"num_write_kb":7950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489378+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:01:48.489378+0000","last_peered":"2024-09-14T07:01:48.489378+0000","last_clean":"2024-09-14T07:01:48.489378+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:01:48.489378+0000","last_undegraded":"2024-09-14T07:01:48.489378+0000","last_fullsized":"2024-09-14T07:01:48.489378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"129'508","reported_seq":3947,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237958+0000","last_change":"2024-09-14T07:01:47.492455+0000","last_active":"2024-09-14T07:01:49.237958+0000","last_peered":"2024-09-14T07:01:49.237958+0000","last_clean":"2024-09-14T07:01:49.237958+0000","last_became_active":"2024-09-14T07:01:47.490786+0000","last_became_peered":"2024-09-14T07:01:47.490786+0000","last_unstale":"2024-09-14T07:01:49.237958+0000","last_undegraded":"2024-09-14T07:01:49.237958+0000","last_fullsized":"2024-09-14T07:01:49.237958+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:11.019913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070556600000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4047,"num_read_kb":4338,"num_write":428,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":305,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489570+0000","last_change":"2024-09-14T07:01:33.416728+0000","last_active":"2024-09-14T07:01:48.489570+0000","last_peered":"2024-09-14T07:01:48.489570+0000","last_clean":"2024-09-14T07:01:48.489570+0000","last_became_active":"2024-09-14T07:01:33.416397+0000","last_became_peered":"2024-09-14T07:01:33.416397+0000","last_unstale":"2024-09-14T07:01:48.489570+0000","last_undegraded":"2024-09-14T07:01:48.489570+0000","last_fullsized":"2024-09-14T07:01:48.489570+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:56.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"132'573","reported_seq":10464,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116353+0000","last_change":"2024-09-14T07:01:36.304148+0000","last_active":"2024-09-14T07:01:49.116353+0000","last_peered":"2024-09-14T07:01:49.116353+0000","last_clean":"2024-09-14T07:01:49.116353+0000","last_became_active":"2024-09-14T07:01:36.302844+0000","last_became_peered":"2024-09-14T07:01:36.302844+0000","last_unstale":"2024-09-14T07:01:49.116353+0000","last_undegraded":"2024-09-14T07:01:49.116353+0000","last_fullsized":"2024-09-14T07:01:49.116353+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:45:52.752571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093736199999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8151,"num_read_kb":12914,"num_write":603,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"133'814","reported_seq":8542,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489516+0000","last_change":"2024-09-14T07:01:35.302802+0000","last_active":"2024-09-14T07:01:48.489516+0000","last_peered":"2024-09-14T07:01:48.489516+0000","last_clean":"2024-09-14T07:01:48.489516+0000","last_became_active":"2024-09-14T07:01:35.301421+0000","last_became_peered":"2024-09-14T07:01:35.301421+0000","last_unstale":"2024-09-14T07:01:48.489516+0000","last_undegraded":"2024-09-14T07:01:48.489516+0000","last_fullsized":"2024-09-14T07:01:48.489516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:41:57.632029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027805429999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":8837,"num_read_kb":11133,"num_write":1016,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":31,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489044+0000","last_change":"2024-09-14T07:01:40.447790+0000","last_active":"2024-09-14T07:01:48.489044+0000","last_peered":"2024-09-14T07:01:48.489044+0000","last_clean":"2024-09-14T07:01:48.489044+0000","last_became_active":"2024-09-14T07:01:40.444161+0000","last_became_peered":"2024-09-14T07:01:40.444161+0000","last_unstale":"2024-09-14T07:01:48.489044+0000","last_undegraded":"2024-09-14T07:01:48.489044+0000","last_fullsized":"2024-09-14T07:01:48.489044+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:14.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"131'477","reported_seq":22200,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489342+0000","last_change":"2024-09-14T07:01:29.368340+0000","last_active":"2024-09-14T07:01:48.489342+0000","last_peered":"2024-09-14T07:01:48.489342+0000","last_clean":"2024-09-14T07:01:48.489342+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:01:48.489342+0000","last_undegraded":"2024-09-14T07:01:48.489342+0000","last_fullsized":"2024-09-14T07:01:48.489342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:44.687681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089711299999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":22342,"num_read_kb":44381,"num_write":481,"num_write_kb":17365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":280,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116984+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:01:49.116984+0000","last_peered":"2024-09-14T07:01:49.116984+0000","last_clean":"2024-09-14T07:01:49.116984+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:01:49.116984+0000","last_undegraded":"2024-09-14T07:01:49.116984+0000","last_fullsized":"2024-09-14T07:01:49.116984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"129'389","reported_seq":2578,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237880+0000","last_change":"2024-09-14T07:01:47.487342+0000","last_active":"2024-09-14T07:01:49.237880+0000","last_peered":"2024-09-14T07:01:49.237880+0000","last_clean":"2024-09-14T07:01:49.237880+0000","last_became_active":"2024-09-14T07:01:47.485661+0000","last_became_peered":"2024-09-14T07:01:47.485661+0000","last_unstale":"2024-09-14T07:01:49.237880+0000","last_undegraded":"2024-09-14T07:01:49.237880+0000","last_fullsized":"2024-09-14T07:01:49.237880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:10.350553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602144,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3106,"num_read_kb":12645,"num_write":311,"num_write_kb":7126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488945+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T07:01:48.488945+0000","last_peered":"2024-09-14T07:01:48.488945+0000","last_clean":"2024-09-14T07:01:48.488945+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:01:48.488945+0000","last_undegraded":"2024-09-14T07:01:48.488945+0000","last_fullsized":"2024-09-14T07:01:48.488945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"132'710","reported_seq":10681,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.299674+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:09.671718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074696400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11440,"num_read_kb":36601,"num_write":924,"num_write_kb":12561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":302,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489125+0000","last_change":"2024-09-14T07:00:27.523385+0000","last_active":"2024-09-14T07:01:48.489125+0000","last_peered":"2024-09-14T07:01:48.489125+0000","last_clean":"2024-09-14T07:01:48.489125+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:01:48.489125+0000","last_undegraded":"2024-09-14T07:01:48.489125+0000","last_fullsized":"2024-09-14T07:01:48.489125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:19.504174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":371,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116500+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T07:01:49.116500+0000","last_peered":"2024-09-14T07:01:49.116500+0000","last_clean":"2024-09-14T07:01:49.116500+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:01:49.116500+0000","last_undegraded":"2024-09-14T07:01:49.116500+0000","last_fullsized":"2024-09-14T07:01:49.116500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"130'594","reported_seq":10088,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116783+0000","last_change":"2024-09-14T07:01:29.297332+0000","last_active":"2024-09-14T07:01:49.116783+0000","last_peered":"2024-09-14T07:01:49.116783+0000","last_clean":"2024-09-14T07:01:49.116783+0000","last_became_active":"2024-09-14T07:00:53.610102+0000","last_became_peered":"2024-09-14T07:00:53.610102+0000","last_unstale":"2024-09-14T07:01:49.116783+0000","last_undegraded":"2024-09-14T07:01:49.116783+0000","last_fullsized":"2024-09-14T07:01:49.116783+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:58.662454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097674599999999991,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":9511,"num_read_kb":20954,"num_write":534,"num_write_kb":10486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4712244,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"133'769","reported_seq":6051,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116875+0000","last_change":"2024-09-14T07:01:37.305171+0000","last_active":"2024-09-14T07:01:49.116875+0000","last_peered":"2024-09-14T07:01:49.116875+0000","last_clean":"2024-09-14T07:01:49.116875+0000","last_became_active":"2024-09-14T07:01:37.303369+0000","last_became_peered":"2024-09-14T07:01:37.303369+0000","last_unstale":"2024-09-14T07:01:49.116875+0000","last_undegraded":"2024-09-14T07:01:49.116875+0000","last_fullsized":"2024-09-14T07:01:49.116875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:52.195099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024576519999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8449,"num_read_kb":18168,"num_write":805,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"132'1123","reported_seq":8414,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489096+0000","last_change":"2024-09-14T07:01:29.295466+0000","last_active":"2024-09-14T07:01:48.489096+0000","last_peered":"2024-09-14T07:01:48.489096+0000","last_clean":"2024-09-14T07:01:48.489096+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:01:48.489096+0000","last_undegraded":"2024-09-14T07:01:48.489096+0000","last_fullsized":"2024-09-14T07:01:48.489096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:23:01.075768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00157712,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":10507,"num_read_kb":9740,"num_write":1539,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'352","reported_seq":3167,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489439+0000","last_change":"2024-09-14T07:01:36.307001+0000","last_active":"2024-09-14T07:01:48.489439+0000","last_peered":"2024-09-14T07:01:48.489439+0000","last_clean":"2024-09-14T07:01:48.489439+0000","last_became_active":"2024-09-14T07:01:36.305298+0000","last_became_peered":"2024-09-14T07:01:36.305298+0000","last_unstale":"2024-09-14T07:01:48.489439+0000","last_undegraded":"2024-09-14T07:01:48.489439+0000","last_fullsized":"2024-09-14T07:01:48.489439+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:38:17.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074805599999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1936,"num_read_kb":8361,"num_write":285,"num_write_kb":3165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"128'460","reported_seq":5812,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489731+0000","last_change":"2024-09-14T07:01:35.303666+0000","last_active":"2024-09-14T07:01:48.489731+0000","last_peered":"2024-09-14T07:01:48.489731+0000","last_clean":"2024-09-14T07:01:48.489731+0000","last_became_active":"2024-09-14T07:01:35.302427+0000","last_became_peered":"2024-09-14T07:01:35.302427+0000","last_unstale":"2024-09-14T07:01:48.489731+0000","last_undegraded":"2024-09-14T07:01:48.489731+0000","last_fullsized":"2024-09-14T07:01:48.489731+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:57.816084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724418,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5156,"num_read_kb":11268,"num_write":417,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"133'441","reported_seq":2271,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489369+0000","last_change":"2024-09-14T07:01:29.295502+0000","last_active":"2024-09-14T07:01:48.489369+0000","last_peered":"2024-09-14T07:01:48.489369+0000","last_clean":"2024-09-14T07:01:48.489369+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:01:48.489369+0000","last_undegraded":"2024-09-14T07:01:48.489369+0000","last_fullsized":"2024-09-14T07:01:48.489369+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:12.777971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015159500000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1920,"num_read_kb":6604,"num_write":451,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"132'450","reported_seq":12399,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116494+0000","last_change":"2024-09-14T07:01:29.344029+0000","last_active":"2024-09-14T07:01:49.116494+0000","last_peered":"2024-09-14T07:01:49.116494+0000","last_clean":"2024-09-14T07:01:49.116494+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:01:49.116494+0000","last_undegraded":"2024-09-14T07:01:49.116494+0000","last_fullsized":"2024-09-14T07:01:49.116494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:57.081265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728802,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":12086,"num_read_kb":54262,"num_write":423,"num_write_kb":18409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"130'507","reported_seq":4859,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489784+0000","last_change":"2024-09-14T07:01:29.346684+0000","last_active":"2024-09-14T07:01:48.489784+0000","last_peered":"2024-09-14T07:01:48.489784+0000","last_clean":"2024-09-14T07:01:48.489784+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:01:48.489784+0000","last_undegraded":"2024-09-14T07:01:48.489784+0000","last_fullsized":"2024-09-14T07:01:48.489784+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:59:29.173513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001321022,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5747,"num_read_kb":11430,"num_write":424,"num_write_kb":4665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"131'440","reported_seq":4097,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116394+0000","last_change":"2024-09-14T07:01:29.295627+0000","last_active":"2024-09-14T07:01:49.116394+0000","last_peered":"2024-09-14T07:01:49.116394+0000","last_clean":"2024-09-14T07:01:49.116394+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:01:49.116394+0000","last_undegraded":"2024-09-14T07:01:49.116394+0000","last_fullsized":"2024-09-14T07:01:49.116394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:58.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095644100000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3634,"num_read_kb":12358,"num_write":404,"num_write_kb":6155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"130'622","reported_seq":11103,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.238002+0000","last_change":"2024-09-14T07:01:47.492336+0000","last_active":"2024-09-14T07:01:49.238002+0000","last_peered":"2024-09-14T07:01:49.238002+0000","last_clean":"2024-09-14T07:01:49.238002+0000","last_became_active":"2024-09-14T07:01:47.490646+0000","last_became_peered":"2024-09-14T07:01:47.490646+0000","last_unstale":"2024-09-14T07:01:49.238002+0000","last_undegraded":"2024-09-14T07:01:49.238002+0000","last_fullsized":"2024-09-14T07:01:49.238002+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:06:20.079036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001614826,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12269,"num_read_kb":24191,"num_write":663,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"131'725","reported_seq":7151,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237968+0000","last_change":"2024-09-14T07:01:47.487409+0000","last_active":"2024-09-14T07:01:49.237968+0000","last_peered":"2024-09-14T07:01:49.237968+0000","last_clean":"2024-09-14T07:01:49.237968+0000","last_became_active":"2024-09-14T07:01:47.485532+0000","last_became_peered":"2024-09-14T07:01:47.485532+0000","last_unstale":"2024-09-14T07:01:49.237968+0000","last_undegraded":"2024-09-14T07:01:49.237968+0000","last_fullsized":"2024-09-14T07:01:49.237968+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:55.605522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071147799999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8637,"num_read_kb":13164,"num_write":610,"num_write_kb":4987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"131'615","reported_seq":6529,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488972+0000","last_change":"2024-09-14T07:01:29.296192+0000","last_active":"2024-09-14T07:01:48.488972+0000","last_peered":"2024-09-14T07:01:48.488972+0000","last_clean":"2024-09-14T07:01:48.488972+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:01:48.488972+0000","last_undegraded":"2024-09-14T07:01:48.488972+0000","last_fullsized":"2024-09-14T07:01:48.488972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:49:30.451851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019616249999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6772,"num_read_kb":10244,"num_write":478,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":579,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":575,"num_read":241540,"num_read_kb":530952,"num_write":22797,"num_write_kb":271683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3188,"num_bytes_recovered":84516668,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20427,"ondisk_log_size":20427,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41276,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330564,"statfs":{"total":96636764160,"available":96594497536,"internally_reserved":0,"allocated":1810432,"data_stored":872232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59344,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312496,"statfs":{"total":96636764160,"available":96575995904,"internally_reserved":0,"allocated":1822720,"data_stored":878219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59332,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312508,"statfs":{"total":96636764160,"available":96576008192,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64948,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306892,"statfs":{"total":96636764160,"available":96570257408,"internally_reserved":0,"allocated":1867776,"data_stored":1462600,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69072,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302768,"statfs":{"total":96636764160,"available":96566034432,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1880064,"data_stored":1468587,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45436,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326404,"statfs":{"total":96636764160,"available":96590237696,"internally_reserved":0,"allocated":1806336,"data_stored":872213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:01:53.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:01:53.370 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:53.371 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:01:53.382 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-14T07:01:51.777824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":575,"num_read":241606,"num_read_kb":531007,"num_write":22900,"num_write_kb":273313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3193,"num_bytes_recovered":84516725,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20502,"ondisk_log_size":20502,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":431780,"kb_used_data":14300,"kb_used_omap":12,"kb_used_meta":417267,"kb_avail":754542940,"statfs":{"total":773094113280,"available":772651970560,"internally_reserved":0,"allocated":14643200,"data_stored":8188451,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":427281969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001513"},"pg_stats":[{"pgid":"3.1f","version":"133'538","reported_seq":7941,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489581+0000","last_change":"2024-09-14T07:01:29.346010+0000","last_active":"2024-09-14T07:01:48.489581+0000","last_peered":"2024-09-14T07:01:48.489581+0000","last_clean":"2024-09-14T07:01:48.489581+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:01:48.489581+0000","last_undegraded":"2024-09-14T07:01:48.489581+0000","last_fullsized":"2024-09-14T07:01:48.489581+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:00.019940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099215999999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8424,"num_read_kb":37327,"num_write":508,"num_write_kb":21659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"130'423","reported_seq":3021,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489455+0000","last_change":"2024-09-14T07:01:29.367999+0000","last_active":"2024-09-14T07:01:48.489455+0000","last_peered":"2024-09-14T07:01:48.489455+0000","last_clean":"2024-09-14T07:01:48.489455+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:01:48.489455+0000","last_undegraded":"2024-09-14T07:01:48.489455+0000","last_fullsized":"2024-09-14T07:01:48.489455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:38.115466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001057501,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2716,"num_read_kb":8402,"num_write":360,"num_write_kb":7998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"130'339","reported_seq":5823,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:43.465257+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T07:01:43.462852+0000","last_became_peered":"2024-09-14T07:01:43.462852+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:36.409964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000797209,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5182,"num_read_kb":12861,"num_write":295,"num_write_kb":5166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"133'2042","reported_seq":4848,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116619+0000","last_change":"2024-09-14T07:01:29.296724+0000","last_active":"2024-09-14T07:01:49.116619+0000","last_peered":"2024-09-14T07:01:49.116619+0000","last_clean":"2024-09-14T07:01:49.116619+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:01:49.116619+0000","last_undegraded":"2024-09-14T07:01:49.116619+0000","last_fullsized":"2024-09-14T07:01:49.116619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":2042,"log_dups_size":0,"ondisk_log_size":2042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:44.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096933700000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11671,"num_read_kb":22543,"num_write":3453,"num_write_kb":11219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"131'900","reported_seq":3379,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116967+0000","last_change":"2024-09-14T07:01:29.299798+0000","last_active":"2024-09-14T07:01:49.116967+0000","last_peered":"2024-09-14T07:01:49.116967+0000","last_clean":"2024-09-14T07:01:49.116967+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:01:49.116967+0000","last_undegraded":"2024-09-14T07:01:49.116967+0000","last_fullsized":"2024-09-14T07:01:49.116967+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:51.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087443700000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4436,"num_read_kb":12231,"num_write":1137,"num_write_kb":20066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"133'1187","reported_seq":8951,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116584+0000","last_change":"2024-09-14T07:01:29.296969+0000","last_active":"2024-09-14T07:01:49.116584+0000","last_peered":"2024-09-14T07:01:49.116584+0000","last_clean":"2024-09-14T07:01:49.116584+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:01:49.116584+0000","last_undegraded":"2024-09-14T07:01:49.116584+0000","last_fullsized":"2024-09-14T07:01:49.116584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:26:48.784038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015092009999999999,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":13058,"num_read_kb":21088,"num_write":1639,"num_write_kb":6895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"133'517","reported_seq":5015,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.300460+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:54:21.579346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00137473,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4742,"num_read_kb":12493,"num_write":520,"num_write_kb":11830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"132'662","reported_seq":3807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:29.346100+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:15.330383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011507799999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5024,"num_read_kb":10270,"num_write":682,"num_write_kb":6392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"130'392","reported_seq":6847,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489521+0000","last_change":"2024-09-14T07:01:29.368091+0000","last_active":"2024-09-14T07:01:48.489521+0000","last_peered":"2024-09-14T07:01:48.489521+0000","last_clean":"2024-09-14T07:01:48.489521+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:01:48.489521+0000","last_undegraded":"2024-09-14T07:01:48.489521+0000","last_fullsized":"2024-09-14T07:01:48.489521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:17.259452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010952469999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7606,"num_read_kb":16770,"num_write":322,"num_write_kb":2846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'413","reported_seq":5194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116357+0000","last_change":"2024-09-14T07:01:43.464245+0000","last_active":"2024-09-14T07:01:49.116357+0000","last_peered":"2024-09-14T07:01:49.116357+0000","last_clean":"2024-09-14T07:01:49.116357+0000","last_became_active":"2024-09-14T07:01:43.462372+0000","last_became_peered":"2024-09-14T07:01:43.462372+0000","last_unstale":"2024-09-14T07:01:49.116357+0000","last_undegraded":"2024-09-14T07:01:49.116357+0000","last_fullsized":"2024-09-14T07:01:49.116357+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:32:17.252003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022563269999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4902,"num_read_kb":8441,"num_write":353,"num_write_kb":3835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"130'346","reported_seq":2783,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489545+0000","last_change":"2024-09-14T07:01:36.306565+0000","last_active":"2024-09-14T07:01:48.489545+0000","last_peered":"2024-09-14T07:01:48.489545+0000","last_clean":"2024-09-14T07:01:48.489545+0000","last_became_active":"2024-09-14T07:01:36.304931+0000","last_became_peered":"2024-09-14T07:01:36.304931+0000","last_unstale":"2024-09-14T07:01:48.489545+0000","last_undegraded":"2024-09-14T07:01:48.489545+0000","last_fullsized":"2024-09-14T07:01:48.489545+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:34.355169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020075470000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1776,"num_read_kb":3899,"num_write":305,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"131'789","reported_seq":5674,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489195+0000","last_change":"2024-09-14T07:01:43.465640+0000","last_active":"2024-09-14T07:01:48.489195+0000","last_peered":"2024-09-14T07:01:48.489195+0000","last_clean":"2024-09-14T07:01:48.489195+0000","last_became_active":"2024-09-14T07:01:43.461464+0000","last_became_peered":"2024-09-14T07:01:43.461464+0000","last_unstale":"2024-09-14T07:01:48.489195+0000","last_undegraded":"2024-09-14T07:01:48.489195+0000","last_fullsized":"2024-09-14T07:01:48.489195+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:07.293229+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0087910980000000007,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6730,"num_read_kb":12616,"num_write":1057,"num_write_kb":7169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489294+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:01:48.489294+0000","last_peered":"2024-09-14T07:01:48.489294+0000","last_clean":"2024-09-14T07:01:48.489294+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:01:48.489294+0000","last_undegraded":"2024-09-14T07:01:48.489294+0000","last_fullsized":"2024-09-14T07:01:48.489294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"133'800","reported_seq":7023,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116939+0000","last_change":"2024-09-14T07:01:29.300111+0000","last_active":"2024-09-14T07:01:49.116939+0000","last_peered":"2024-09-14T07:01:49.116939+0000","last_clean":"2024-09-14T07:01:49.116939+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:01:49.116939+0000","last_undegraded":"2024-09-14T07:01:49.116939+0000","last_fullsized":"2024-09-14T07:01:49.116939+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:46.258973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010359880000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8046,"num_read_kb":9156,"num_write":887,"num_write_kb":5801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.117092+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:01:49.117092+0000","last_peered":"2024-09-14T07:01:49.117092+0000","last_clean":"2024-09-14T07:01:49.117092+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:01:49.117092+0000","last_undegraded":"2024-09-14T07:01:49.117092+0000","last_fullsized":"2024-09-14T07:01:49.117092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"132'510","reported_seq":11748,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116991+0000","last_change":"2024-09-14T07:01:36.306671+0000","last_active":"2024-09-14T07:01:49.116991+0000","last_peered":"2024-09-14T07:01:49.116991+0000","last_clean":"2024-09-14T07:01:49.116991+0000","last_became_active":"2024-09-14T07:01:36.305222+0000","last_became_peered":"2024-09-14T07:01:36.305222+0000","last_unstale":"2024-09-14T07:01:49.116991+0000","last_undegraded":"2024-09-14T07:01:49.116991+0000","last_fullsized":"2024-09-14T07:01:49.116991+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:12.004745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001033576,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12680,"num_read_kb":20099,"num_write":483,"num_write_kb":7950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489378+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:01:48.489378+0000","last_peered":"2024-09-14T07:01:48.489378+0000","last_clean":"2024-09-14T07:01:48.489378+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:01:48.489378+0000","last_undegraded":"2024-09-14T07:01:48.489378+0000","last_fullsized":"2024-09-14T07:01:48.489378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"129'508","reported_seq":3947,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237958+0000","last_change":"2024-09-14T07:01:47.492455+0000","last_active":"2024-09-14T07:01:49.237958+0000","last_peered":"2024-09-14T07:01:49.237958+0000","last_clean":"2024-09-14T07:01:49.237958+0000","last_became_active":"2024-09-14T07:01:47.490786+0000","last_became_peered":"2024-09-14T07:01:47.490786+0000","last_unstale":"2024-09-14T07:01:49.237958+0000","last_undegraded":"2024-09-14T07:01:49.237958+0000","last_fullsized":"2024-09-14T07:01:49.237958+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:11.019913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070556600000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4047,"num_read_kb":4338,"num_write":428,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":305,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489570+0000","last_change":"2024-09-14T07:01:33.416728+0000","last_active":"2024-09-14T07:01:48.489570+0000","last_peered":"2024-09-14T07:01:48.489570+0000","last_clean":"2024-09-14T07:01:48.489570+0000","last_became_active":"2024-09-14T07:01:33.416397+0000","last_became_peered":"2024-09-14T07:01:33.416397+0000","last_unstale":"2024-09-14T07:01:48.489570+0000","last_undegraded":"2024-09-14T07:01:48.489570+0000","last_fullsized":"2024-09-14T07:01:48.489570+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:56.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"132'573","reported_seq":10464,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116353+0000","last_change":"2024-09-14T07:01:36.304148+0000","last_active":"2024-09-14T07:01:49.116353+0000","last_peered":"2024-09-14T07:01:49.116353+0000","last_clean":"2024-09-14T07:01:49.116353+0000","last_became_active":"2024-09-14T07:01:36.302844+0000","last_became_peered":"2024-09-14T07:01:36.302844+0000","last_unstale":"2024-09-14T07:01:49.116353+0000","last_undegraded":"2024-09-14T07:01:49.116353+0000","last_fullsized":"2024-09-14T07:01:49.116353+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:45:52.752571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093736199999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8151,"num_read_kb":12914,"num_write":603,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"133'814","reported_seq":8542,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489516+0000","last_change":"2024-09-14T07:01:35.302802+0000","last_active":"2024-09-14T07:01:48.489516+0000","last_peered":"2024-09-14T07:01:48.489516+0000","last_clean":"2024-09-14T07:01:48.489516+0000","last_became_active":"2024-09-14T07:01:35.301421+0000","last_became_peered":"2024-09-14T07:01:35.301421+0000","last_unstale":"2024-09-14T07:01:48.489516+0000","last_undegraded":"2024-09-14T07:01:48.489516+0000","last_fullsized":"2024-09-14T07:01:48.489516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:41:57.632029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027805429999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":8837,"num_read_kb":11133,"num_write":1016,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":31,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489044+0000","last_change":"2024-09-14T07:01:40.447790+0000","last_active":"2024-09-14T07:01:48.489044+0000","last_peered":"2024-09-14T07:01:48.489044+0000","last_clean":"2024-09-14T07:01:48.489044+0000","last_became_active":"2024-09-14T07:01:40.444161+0000","last_became_peered":"2024-09-14T07:01:40.444161+0000","last_unstale":"2024-09-14T07:01:48.489044+0000","last_undegraded":"2024-09-14T07:01:48.489044+0000","last_fullsized":"2024-09-14T07:01:48.489044+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:14.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"131'477","reported_seq":22200,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489342+0000","last_change":"2024-09-14T07:01:29.368340+0000","last_active":"2024-09-14T07:01:48.489342+0000","last_peered":"2024-09-14T07:01:48.489342+0000","last_clean":"2024-09-14T07:01:48.489342+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:01:48.489342+0000","last_undegraded":"2024-09-14T07:01:48.489342+0000","last_fullsized":"2024-09-14T07:01:48.489342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:44.687681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089711299999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":22342,"num_read_kb":44381,"num_write":481,"num_write_kb":17365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":280,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116984+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:01:49.116984+0000","last_peered":"2024-09-14T07:01:49.116984+0000","last_clean":"2024-09-14T07:01:49.116984+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:01:49.116984+0000","last_undegraded":"2024-09-14T07:01:49.116984+0000","last_fullsized":"2024-09-14T07:01:49.116984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"129'389","reported_seq":2578,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237880+0000","last_change":"2024-09-14T07:01:47.487342+0000","last_active":"2024-09-14T07:01:49.237880+0000","last_peered":"2024-09-14T07:01:49.237880+0000","last_clean":"2024-09-14T07:01:49.237880+0000","last_became_active":"2024-09-14T07:01:47.485661+0000","last_became_peered":"2024-09-14T07:01:47.485661+0000","last_unstale":"2024-09-14T07:01:49.237880+0000","last_undegraded":"2024-09-14T07:01:49.237880+0000","last_fullsized":"2024-09-14T07:01:49.237880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:10.350553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602144,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3106,"num_read_kb":12645,"num_write":311,"num_write_kb":7126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488945+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T07:01:48.488945+0000","last_peered":"2024-09-14T07:01:48.488945+0000","last_clean":"2024-09-14T07:01:48.488945+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:01:48.488945+0000","last_undegraded":"2024-09-14T07:01:48.488945+0000","last_fullsized":"2024-09-14T07:01:48.488945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"132'710","reported_seq":10681,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.299674+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:09.671718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074696400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11440,"num_read_kb":36601,"num_write":924,"num_write_kb":12561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":302,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489125+0000","last_change":"2024-09-14T07:00:27.523385+0000","last_active":"2024-09-14T07:01:48.489125+0000","last_peered":"2024-09-14T07:01:48.489125+0000","last_clean":"2024-09-14T07:01:48.489125+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:01:48.489125+0000","last_undegraded":"2024-09-14T07:01:48.489125+0000","last_fullsized":"2024-09-14T07:01:48.489125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:19.504174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":371,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116500+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T07:01:49.116500+0000","last_peered":"2024-09-14T07:01:49.116500+0000","last_clean":"2024-09-14T07:01:49.116500+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:01:49.116500+0000","last_undegraded":"2024-09-14T07:01:49.116500+0000","last_fullsized":"2024-09-14T07:01:49.116500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"130'594","reported_seq":10088,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116783+0000","last_change":"2024-09-14T07:01:29.297332+0000","last_active":"2024-09-14T07:01:49.116783+0000","last_peered":"2024-09-14T07:01:49.116783+0000","last_clean":"2024-09-14T07:01:49.116783+0000","last_became_active":"2024-09-14T07:00:53.610102+0000","last_became_peered":"2024-09-14T07:00:53.610102+0000","last_unstale":"2024-09-14T07:01:49.116783+0000","last_undegraded":"2024-09-14T07:01:49.116783+0000","last_fullsized":"2024-09-14T07:01:49.116783+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:58.662454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097674599999999991,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":9511,"num_read_kb":20954,"num_write":534,"num_write_kb":10486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4712244,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"133'769","reported_seq":6051,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116875+0000","last_change":"2024-09-14T07:01:37.305171+0000","last_active":"2024-09-14T07:01:49.116875+0000","last_peered":"2024-09-14T07:01:49.116875+0000","last_clean":"2024-09-14T07:01:49.116875+0000","last_became_active":"2024-09-14T07:01:37.303369+0000","last_became_peered":"2024-09-14T07:01:37.303369+0000","last_unstale":"2024-09-14T07:01:49.116875+0000","last_undegraded":"2024-09-14T07:01:49.116875+0000","last_fullsized":"2024-09-14T07:01:49.116875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:52.195099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024576519999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8449,"num_read_kb":18168,"num_write":805,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"132'1123","reported_seq":8414,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489096+0000","last_change":"2024-09-14T07:01:29.295466+0000","last_active":"2024-09-14T07:01:48.489096+0000","last_peered":"2024-09-14T07:01:48.489096+0000","last_clean":"2024-09-14T07:01:48.489096+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:01:48.489096+0000","last_undegraded":"2024-09-14T07:01:48.489096+0000","last_fullsized":"2024-09-14T07:01:48.489096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:23:01.075768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00157712,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":10507,"num_read_kb":9740,"num_write":1539,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'352","reported_seq":3167,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489439+0000","last_change":"2024-09-14T07:01:36.307001+0000","last_active":"2024-09-14T07:01:48.489439+0000","last_peered":"2024-09-14T07:01:48.489439+0000","last_clean":"2024-09-14T07:01:48.489439+0000","last_became_active":"2024-09-14T07:01:36.305298+0000","last_became_peered":"2024-09-14T07:01:36.305298+0000","last_unstale":"2024-09-14T07:01:48.489439+0000","last_undegraded":"2024-09-14T07:01:48.489439+0000","last_fullsized":"2024-09-14T07:01:48.489439+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:38:17.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074805599999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1936,"num_read_kb":8361,"num_write":285,"num_write_kb":3165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"128'460","reported_seq":5812,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489731+0000","last_change":"2024-09-14T07:01:35.303666+0000","last_active":"2024-09-14T07:01:48.489731+0000","last_peered":"2024-09-14T07:01:48.489731+0000","last_clean":"2024-09-14T07:01:48.489731+0000","last_became_active":"2024-09-14T07:01:35.302427+0000","last_became_peered":"2024-09-14T07:01:35.302427+0000","last_unstale":"2024-09-14T07:01:48.489731+0000","last_undegraded":"2024-09-14T07:01:48.489731+0000","last_fullsized":"2024-09-14T07:01:48.489731+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:57.816084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724418,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5156,"num_read_kb":11268,"num_write":417,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"133'441","reported_seq":2271,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489369+0000","last_change":"2024-09-14T07:01:29.295502+0000","last_active":"2024-09-14T07:01:48.489369+0000","last_peered":"2024-09-14T07:01:48.489369+0000","last_clean":"2024-09-14T07:01:48.489369+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:01:48.489369+0000","last_undegraded":"2024-09-14T07:01:48.489369+0000","last_fullsized":"2024-09-14T07:01:48.489369+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:12.777971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015159500000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1920,"num_read_kb":6604,"num_write":451,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"132'450","reported_seq":12399,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116494+0000","last_change":"2024-09-14T07:01:29.344029+0000","last_active":"2024-09-14T07:01:49.116494+0000","last_peered":"2024-09-14T07:01:49.116494+0000","last_clean":"2024-09-14T07:01:49.116494+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:01:49.116494+0000","last_undegraded":"2024-09-14T07:01:49.116494+0000","last_fullsized":"2024-09-14T07:01:49.116494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:57.081265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728802,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":12086,"num_read_kb":54262,"num_write":423,"num_write_kb":18409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"130'507","reported_seq":4859,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489784+0000","last_change":"2024-09-14T07:01:29.346684+0000","last_active":"2024-09-14T07:01:48.489784+0000","last_peered":"2024-09-14T07:01:48.489784+0000","last_clean":"2024-09-14T07:01:48.489784+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:01:48.489784+0000","last_undegraded":"2024-09-14T07:01:48.489784+0000","last_fullsized":"2024-09-14T07:01:48.489784+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:59:29.173513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001321022,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5747,"num_read_kb":11430,"num_write":424,"num_write_kb":4665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"131'440","reported_seq":4097,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116394+0000","last_change":"2024-09-14T07:01:29.295627+0000","last_active":"2024-09-14T07:01:49.116394+0000","last_peered":"2024-09-14T07:01:49.116394+0000","last_clean":"2024-09-14T07:01:49.116394+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:01:49.116394+0000","last_undegraded":"2024-09-14T07:01:49.116394+0000","last_fullsized":"2024-09-14T07:01:49.116394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:58.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095644100000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3634,"num_read_kb":12358,"num_write":404,"num_write_kb":6155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"130'622","reported_seq":11103,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.238002+0000","last_change":"2024-09-14T07:01:47.492336+0000","last_active":"2024-09-14T07:01:49.238002+0000","last_peered":"2024-09-14T07:01:49.238002+0000","last_clean":"2024-09-14T07:01:49.238002+0000","last_became_active":"2024-09-14T07:01:47.490646+0000","last_became_peered":"2024-09-14T07:01:47.490646+0000","last_unstale":"2024-09-14T07:01:49.238002+0000","last_undegraded":"2024-09-14T07:01:49.238002+0000","last_fullsized":"2024-09-14T07:01:49.238002+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:06:20.079036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001614826,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12269,"num_read_kb":24191,"num_write":663,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"131'725","reported_seq":7151,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237968+0000","last_change":"2024-09-14T07:01:47.487409+0000","last_active":"2024-09-14T07:01:49.237968+0000","last_peered":"2024-09-14T07:01:49.237968+0000","last_clean":"2024-09-14T07:01:49.237968+0000","last_became_active":"2024-09-14T07:01:47.485532+0000","last_became_peered":"2024-09-14T07:01:47.485532+0000","last_unstale":"2024-09-14T07:01:49.237968+0000","last_undegraded":"2024-09-14T07:01:49.237968+0000","last_fullsized":"2024-09-14T07:01:49.237968+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:55.605522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071147799999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8637,"num_read_kb":13164,"num_write":610,"num_write_kb":4987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"131'615","reported_seq":6529,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488972+0000","last_change":"2024-09-14T07:01:29.296192+0000","last_active":"2024-09-14T07:01:48.488972+0000","last_peered":"2024-09-14T07:01:48.488972+0000","last_clean":"2024-09-14T07:01:48.488972+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:01:48.488972+0000","last_undegraded":"2024-09-14T07:01:48.488972+0000","last_fullsized":"2024-09-14T07:01:48.488972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:49:30.451851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019616249999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6772,"num_read_kb":10244,"num_write":478,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":579,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":575,"num_read":241540,"num_read_kb":530952,"num_write":22797,"num_write_kb":271683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3188,"num_bytes_recovered":84516668,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20427,"ondisk_log_size":20427,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41276,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330564,"statfs":{"total":96636764160,"available":96594497536,"internally_reserved":0,"allocated":1810432,"data_stored":872232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59344,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312496,"statfs":{"total":96636764160,"available":96575995904,"internally_reserved":0,"allocated":1822720,"data_stored":878219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59332,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312508,"statfs":{"total":96636764160,"available":96576008192,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64948,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306892,"statfs":{"total":96636764160,"available":96570257408,"internally_reserved":0,"allocated":1867776,"data_stored":1462600,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69072,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302768,"statfs":{"total":96636764160,"available":96566034432,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1880064,"data_stored":1468587,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45436,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326404,"statfs":{"total":96636764160,"available":96590237696,"internally_reserved":0,"allocated":1806336,"data_stored":872213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:01:53.384 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T07:01:53.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:01:53.683 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:53.683 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:01:53.696 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-14T07:01:51.777824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":575,"num_read":241606,"num_read_kb":531007,"num_write":22900,"num_write_kb":273313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3193,"num_bytes_recovered":84516725,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20502,"ondisk_log_size":20502,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":431780,"kb_used_data":14300,"kb_used_omap":12,"kb_used_meta":417267,"kb_avail":754542940,"statfs":{"total":773094113280,"available":772651970560,"internally_reserved":0,"allocated":14643200,"data_stored":8188451,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":427281969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-348,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001513"},"pg_stats":[{"pgid":"3.1f","version":"133'538","reported_seq":7941,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489581+0000","last_change":"2024-09-14T07:01:29.346010+0000","last_active":"2024-09-14T07:01:48.489581+0000","last_peered":"2024-09-14T07:01:48.489581+0000","last_clean":"2024-09-14T07:01:48.489581+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:01:48.489581+0000","last_undegraded":"2024-09-14T07:01:48.489581+0000","last_fullsized":"2024-09-14T07:01:48.489581+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:00.019940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099215999999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8424,"num_read_kb":37327,"num_write":508,"num_write_kb":21659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"130'423","reported_seq":3021,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489455+0000","last_change":"2024-09-14T07:01:29.367999+0000","last_active":"2024-09-14T07:01:48.489455+0000","last_peered":"2024-09-14T07:01:48.489455+0000","last_clean":"2024-09-14T07:01:48.489455+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:01:48.489455+0000","last_undegraded":"2024-09-14T07:01:48.489455+0000","last_fullsized":"2024-09-14T07:01:48.489455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:38.115466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001057501,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2716,"num_read_kb":8402,"num_write":360,"num_write_kb":7998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"130'339","reported_seq":5823,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:43.465257+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T07:01:43.462852+0000","last_became_peered":"2024-09-14T07:01:43.462852+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:36.409964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000797209,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5182,"num_read_kb":12861,"num_write":295,"num_write_kb":5166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"133'2042","reported_seq":4848,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116619+0000","last_change":"2024-09-14T07:01:29.296724+0000","last_active":"2024-09-14T07:01:49.116619+0000","last_peered":"2024-09-14T07:01:49.116619+0000","last_clean":"2024-09-14T07:01:49.116619+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:01:49.116619+0000","last_undegraded":"2024-09-14T07:01:49.116619+0000","last_fullsized":"2024-09-14T07:01:49.116619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":2042,"log_dups_size":0,"ondisk_log_size":2042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:44.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096933700000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11671,"num_read_kb":22543,"num_write":3453,"num_write_kb":11219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"131'900","reported_seq":3379,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116967+0000","last_change":"2024-09-14T07:01:29.299798+0000","last_active":"2024-09-14T07:01:49.116967+0000","last_peered":"2024-09-14T07:01:49.116967+0000","last_clean":"2024-09-14T07:01:49.116967+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:01:49.116967+0000","last_undegraded":"2024-09-14T07:01:49.116967+0000","last_fullsized":"2024-09-14T07:01:49.116967+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:51.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087443700000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4436,"num_read_kb":12231,"num_write":1137,"num_write_kb":20066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"133'1187","reported_seq":8951,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116584+0000","last_change":"2024-09-14T07:01:29.296969+0000","last_active":"2024-09-14T07:01:49.116584+0000","last_peered":"2024-09-14T07:01:49.116584+0000","last_clean":"2024-09-14T07:01:49.116584+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:01:49.116584+0000","last_undegraded":"2024-09-14T07:01:49.116584+0000","last_fullsized":"2024-09-14T07:01:49.116584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:26:48.784038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015092009999999999,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":13058,"num_read_kb":21088,"num_write":1639,"num_write_kb":6895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"133'517","reported_seq":5015,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.300460+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:54:21.579346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00137473,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4742,"num_read_kb":12493,"num_write":520,"num_write_kb":11830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"132'662","reported_seq":3807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:29.346100+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:15.330383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011507799999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5024,"num_read_kb":10270,"num_write":682,"num_write_kb":6392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"130'392","reported_seq":6847,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489521+0000","last_change":"2024-09-14T07:01:29.368091+0000","last_active":"2024-09-14T07:01:48.489521+0000","last_peered":"2024-09-14T07:01:48.489521+0000","last_clean":"2024-09-14T07:01:48.489521+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:01:48.489521+0000","last_undegraded":"2024-09-14T07:01:48.489521+0000","last_fullsized":"2024-09-14T07:01:48.489521+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:17.259452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010952469999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7606,"num_read_kb":16770,"num_write":322,"num_write_kb":2846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'413","reported_seq":5194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116357+0000","last_change":"2024-09-14T07:01:43.464245+0000","last_active":"2024-09-14T07:01:49.116357+0000","last_peered":"2024-09-14T07:01:49.116357+0000","last_clean":"2024-09-14T07:01:49.116357+0000","last_became_active":"2024-09-14T07:01:43.462372+0000","last_became_peered":"2024-09-14T07:01:43.462372+0000","last_unstale":"2024-09-14T07:01:49.116357+0000","last_undegraded":"2024-09-14T07:01:49.116357+0000","last_fullsized":"2024-09-14T07:01:49.116357+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:32:17.252003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022563269999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4902,"num_read_kb":8441,"num_write":353,"num_write_kb":3835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"130'346","reported_seq":2783,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489545+0000","last_change":"2024-09-14T07:01:36.306565+0000","last_active":"2024-09-14T07:01:48.489545+0000","last_peered":"2024-09-14T07:01:48.489545+0000","last_clean":"2024-09-14T07:01:48.489545+0000","last_became_active":"2024-09-14T07:01:36.304931+0000","last_became_peered":"2024-09-14T07:01:36.304931+0000","last_unstale":"2024-09-14T07:01:48.489545+0000","last_undegraded":"2024-09-14T07:01:48.489545+0000","last_fullsized":"2024-09-14T07:01:48.489545+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:35:34.355169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020075470000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1776,"num_read_kb":3899,"num_write":305,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"131'789","reported_seq":5674,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489195+0000","last_change":"2024-09-14T07:01:43.465640+0000","last_active":"2024-09-14T07:01:48.489195+0000","last_peered":"2024-09-14T07:01:48.489195+0000","last_clean":"2024-09-14T07:01:48.489195+0000","last_became_active":"2024-09-14T07:01:43.461464+0000","last_became_peered":"2024-09-14T07:01:43.461464+0000","last_unstale":"2024-09-14T07:01:48.489195+0000","last_undegraded":"2024-09-14T07:01:48.489195+0000","last_fullsized":"2024-09-14T07:01:48.489195+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:07.293229+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0087910980000000007,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6730,"num_read_kb":12616,"num_write":1057,"num_write_kb":7169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489294+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:01:48.489294+0000","last_peered":"2024-09-14T07:01:48.489294+0000","last_clean":"2024-09-14T07:01:48.489294+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:01:48.489294+0000","last_undegraded":"2024-09-14T07:01:48.489294+0000","last_fullsized":"2024-09-14T07:01:48.489294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"133'800","reported_seq":7023,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116939+0000","last_change":"2024-09-14T07:01:29.300111+0000","last_active":"2024-09-14T07:01:49.116939+0000","last_peered":"2024-09-14T07:01:49.116939+0000","last_clean":"2024-09-14T07:01:49.116939+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:01:49.116939+0000","last_undegraded":"2024-09-14T07:01:49.116939+0000","last_fullsized":"2024-09-14T07:01:49.116939+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:46.258973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010359880000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8046,"num_read_kb":9156,"num_write":887,"num_write_kb":5801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.117092+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:01:49.117092+0000","last_peered":"2024-09-14T07:01:49.117092+0000","last_clean":"2024-09-14T07:01:49.117092+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:01:49.117092+0000","last_undegraded":"2024-09-14T07:01:49.117092+0000","last_fullsized":"2024-09-14T07:01:49.117092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"132'510","reported_seq":11748,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116991+0000","last_change":"2024-09-14T07:01:36.306671+0000","last_active":"2024-09-14T07:01:49.116991+0000","last_peered":"2024-09-14T07:01:49.116991+0000","last_clean":"2024-09-14T07:01:49.116991+0000","last_became_active":"2024-09-14T07:01:36.305222+0000","last_became_peered":"2024-09-14T07:01:36.305222+0000","last_unstale":"2024-09-14T07:01:49.116991+0000","last_undegraded":"2024-09-14T07:01:49.116991+0000","last_fullsized":"2024-09-14T07:01:49.116991+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:12.004745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001033576,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12680,"num_read_kb":20099,"num_write":483,"num_write_kb":7950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489378+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:01:48.489378+0000","last_peered":"2024-09-14T07:01:48.489378+0000","last_clean":"2024-09-14T07:01:48.489378+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:01:48.489378+0000","last_undegraded":"2024-09-14T07:01:48.489378+0000","last_fullsized":"2024-09-14T07:01:48.489378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"129'508","reported_seq":3947,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237958+0000","last_change":"2024-09-14T07:01:47.492455+0000","last_active":"2024-09-14T07:01:49.237958+0000","last_peered":"2024-09-14T07:01:49.237958+0000","last_clean":"2024-09-14T07:01:49.237958+0000","last_became_active":"2024-09-14T07:01:47.490786+0000","last_became_peered":"2024-09-14T07:01:47.490786+0000","last_unstale":"2024-09-14T07:01:49.237958+0000","last_undegraded":"2024-09-14T07:01:49.237958+0000","last_fullsized":"2024-09-14T07:01:49.237958+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:11.019913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070556600000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4047,"num_read_kb":4338,"num_write":428,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":305,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489570+0000","last_change":"2024-09-14T07:01:33.416728+0000","last_active":"2024-09-14T07:01:48.489570+0000","last_peered":"2024-09-14T07:01:48.489570+0000","last_clean":"2024-09-14T07:01:48.489570+0000","last_became_active":"2024-09-14T07:01:33.416397+0000","last_became_peered":"2024-09-14T07:01:33.416397+0000","last_unstale":"2024-09-14T07:01:48.489570+0000","last_undegraded":"2024-09-14T07:01:48.489570+0000","last_fullsized":"2024-09-14T07:01:48.489570+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:56.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"132'573","reported_seq":10464,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116353+0000","last_change":"2024-09-14T07:01:36.304148+0000","last_active":"2024-09-14T07:01:49.116353+0000","last_peered":"2024-09-14T07:01:49.116353+0000","last_clean":"2024-09-14T07:01:49.116353+0000","last_became_active":"2024-09-14T07:01:36.302844+0000","last_became_peered":"2024-09-14T07:01:36.302844+0000","last_unstale":"2024-09-14T07:01:49.116353+0000","last_undegraded":"2024-09-14T07:01:49.116353+0000","last_fullsized":"2024-09-14T07:01:49.116353+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:45:52.752571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093736199999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8151,"num_read_kb":12914,"num_write":603,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"133'814","reported_seq":8542,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489516+0000","last_change":"2024-09-14T07:01:35.302802+0000","last_active":"2024-09-14T07:01:48.489516+0000","last_peered":"2024-09-14T07:01:48.489516+0000","last_clean":"2024-09-14T07:01:48.489516+0000","last_became_active":"2024-09-14T07:01:35.301421+0000","last_became_peered":"2024-09-14T07:01:35.301421+0000","last_unstale":"2024-09-14T07:01:48.489516+0000","last_undegraded":"2024-09-14T07:01:48.489516+0000","last_fullsized":"2024-09-14T07:01:48.489516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:41:57.632029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027805429999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":8837,"num_read_kb":11133,"num_write":1016,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":31,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489044+0000","last_change":"2024-09-14T07:01:40.447790+0000","last_active":"2024-09-14T07:01:48.489044+0000","last_peered":"2024-09-14T07:01:48.489044+0000","last_clean":"2024-09-14T07:01:48.489044+0000","last_became_active":"2024-09-14T07:01:40.444161+0000","last_became_peered":"2024-09-14T07:01:40.444161+0000","last_unstale":"2024-09-14T07:01:48.489044+0000","last_undegraded":"2024-09-14T07:01:48.489044+0000","last_fullsized":"2024-09-14T07:01:48.489044+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:14.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"131'477","reported_seq":22200,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489342+0000","last_change":"2024-09-14T07:01:29.368340+0000","last_active":"2024-09-14T07:01:48.489342+0000","last_peered":"2024-09-14T07:01:48.489342+0000","last_clean":"2024-09-14T07:01:48.489342+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:01:48.489342+0000","last_undegraded":"2024-09-14T07:01:48.489342+0000","last_fullsized":"2024-09-14T07:01:48.489342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:44.687681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089711299999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":22342,"num_read_kb":44381,"num_write":481,"num_write_kb":17365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":280,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116984+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:01:49.116984+0000","last_peered":"2024-09-14T07:01:49.116984+0000","last_clean":"2024-09-14T07:01:49.116984+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:01:49.116984+0000","last_undegraded":"2024-09-14T07:01:49.116984+0000","last_fullsized":"2024-09-14T07:01:49.116984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"129'389","reported_seq":2578,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237880+0000","last_change":"2024-09-14T07:01:47.487342+0000","last_active":"2024-09-14T07:01:49.237880+0000","last_peered":"2024-09-14T07:01:49.237880+0000","last_clean":"2024-09-14T07:01:49.237880+0000","last_became_active":"2024-09-14T07:01:47.485661+0000","last_became_peered":"2024-09-14T07:01:47.485661+0000","last_unstale":"2024-09-14T07:01:49.237880+0000","last_undegraded":"2024-09-14T07:01:49.237880+0000","last_fullsized":"2024-09-14T07:01:49.237880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:10.350553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602144,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3106,"num_read_kb":12645,"num_write":311,"num_write_kb":7126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488945+0000","last_change":"2024-09-14T06:59:21.766791+0000","last_active":"2024-09-14T07:01:48.488945+0000","last_peered":"2024-09-14T07:01:48.488945+0000","last_clean":"2024-09-14T07:01:48.488945+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:01:48.488945+0000","last_undegraded":"2024-09-14T07:01:48.488945+0000","last_fullsized":"2024-09-14T07:01:48.488945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:54.243847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"132'710","reported_seq":10681,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.299674+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:09.671718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074696400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11440,"num_read_kb":36601,"num_write":924,"num_write_kb":12561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":302,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489125+0000","last_change":"2024-09-14T07:00:27.523385+0000","last_active":"2024-09-14T07:01:48.489125+0000","last_peered":"2024-09-14T07:01:48.489125+0000","last_clean":"2024-09-14T07:01:48.489125+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:01:48.489125+0000","last_undegraded":"2024-09-14T07:01:48.489125+0000","last_fullsized":"2024-09-14T07:01:48.489125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:19.504174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":371,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116500+0000","last_change":"2024-09-14T06:59:14.303064+0000","last_active":"2024-09-14T07:01:49.116500+0000","last_peered":"2024-09-14T07:01:49.116500+0000","last_clean":"2024-09-14T07:01:49.116500+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:01:49.116500+0000","last_undegraded":"2024-09-14T07:01:49.116500+0000","last_fullsized":"2024-09-14T07:01:49.116500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:12.720526+0000","last_clean_scrub_stamp":"2024-09-14T06:59:12.720526+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:15.069194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"130'594","reported_seq":10088,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116783+0000","last_change":"2024-09-14T07:01:29.297332+0000","last_active":"2024-09-14T07:01:49.116783+0000","last_peered":"2024-09-14T07:01:49.116783+0000","last_clean":"2024-09-14T07:01:49.116783+0000","last_became_active":"2024-09-14T07:00:53.610102+0000","last_became_peered":"2024-09-14T07:00:53.610102+0000","last_unstale":"2024-09-14T07:01:49.116783+0000","last_undegraded":"2024-09-14T07:01:49.116783+0000","last_fullsized":"2024-09-14T07:01:49.116783+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:58.662454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097674599999999991,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":9511,"num_read_kb":20954,"num_write":534,"num_write_kb":10486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4712244,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"133'769","reported_seq":6051,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116875+0000","last_change":"2024-09-14T07:01:37.305171+0000","last_active":"2024-09-14T07:01:49.116875+0000","last_peered":"2024-09-14T07:01:49.116875+0000","last_clean":"2024-09-14T07:01:49.116875+0000","last_became_active":"2024-09-14T07:01:37.303369+0000","last_became_peered":"2024-09-14T07:01:37.303369+0000","last_unstale":"2024-09-14T07:01:49.116875+0000","last_undegraded":"2024-09-14T07:01:49.116875+0000","last_fullsized":"2024-09-14T07:01:49.116875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:52.195099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024576519999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8449,"num_read_kb":18168,"num_write":805,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"132'1123","reported_seq":8414,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489096+0000","last_change":"2024-09-14T07:01:29.295466+0000","last_active":"2024-09-14T07:01:48.489096+0000","last_peered":"2024-09-14T07:01:48.489096+0000","last_clean":"2024-09-14T07:01:48.489096+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:01:48.489096+0000","last_undegraded":"2024-09-14T07:01:48.489096+0000","last_fullsized":"2024-09-14T07:01:48.489096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:23:01.075768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00157712,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":10507,"num_read_kb":9740,"num_write":1539,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'352","reported_seq":3167,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489439+0000","last_change":"2024-09-14T07:01:36.307001+0000","last_active":"2024-09-14T07:01:48.489439+0000","last_peered":"2024-09-14T07:01:48.489439+0000","last_clean":"2024-09-14T07:01:48.489439+0000","last_became_active":"2024-09-14T07:01:36.305298+0000","last_became_peered":"2024-09-14T07:01:36.305298+0000","last_unstale":"2024-09-14T07:01:48.489439+0000","last_undegraded":"2024-09-14T07:01:48.489439+0000","last_fullsized":"2024-09-14T07:01:48.489439+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:38:17.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074805599999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1936,"num_read_kb":8361,"num_write":285,"num_write_kb":3165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"128'460","reported_seq":5812,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489731+0000","last_change":"2024-09-14T07:01:35.303666+0000","last_active":"2024-09-14T07:01:48.489731+0000","last_peered":"2024-09-14T07:01:48.489731+0000","last_clean":"2024-09-14T07:01:48.489731+0000","last_became_active":"2024-09-14T07:01:35.302427+0000","last_became_peered":"2024-09-14T07:01:35.302427+0000","last_unstale":"2024-09-14T07:01:48.489731+0000","last_undegraded":"2024-09-14T07:01:48.489731+0000","last_fullsized":"2024-09-14T07:01:48.489731+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:57.816084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724418,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5156,"num_read_kb":11268,"num_write":417,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"133'441","reported_seq":2271,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489369+0000","last_change":"2024-09-14T07:01:29.295502+0000","last_active":"2024-09-14T07:01:48.489369+0000","last_peered":"2024-09-14T07:01:48.489369+0000","last_clean":"2024-09-14T07:01:48.489369+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:01:48.489369+0000","last_undegraded":"2024-09-14T07:01:48.489369+0000","last_fullsized":"2024-09-14T07:01:48.489369+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:12.777971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015159500000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1920,"num_read_kb":6604,"num_write":451,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"132'450","reported_seq":12399,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116494+0000","last_change":"2024-09-14T07:01:29.344029+0000","last_active":"2024-09-14T07:01:49.116494+0000","last_peered":"2024-09-14T07:01:49.116494+0000","last_clean":"2024-09-14T07:01:49.116494+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:01:49.116494+0000","last_undegraded":"2024-09-14T07:01:49.116494+0000","last_fullsized":"2024-09-14T07:01:49.116494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:57.081265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728802,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":12086,"num_read_kb":54262,"num_write":423,"num_write_kb":18409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"130'507","reported_seq":4859,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489784+0000","last_change":"2024-09-14T07:01:29.346684+0000","last_active":"2024-09-14T07:01:48.489784+0000","last_peered":"2024-09-14T07:01:48.489784+0000","last_clean":"2024-09-14T07:01:48.489784+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:01:48.489784+0000","last_undegraded":"2024-09-14T07:01:48.489784+0000","last_fullsized":"2024-09-14T07:01:48.489784+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:59:29.173513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001321022,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5747,"num_read_kb":11430,"num_write":424,"num_write_kb":4665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"131'440","reported_seq":4097,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116394+0000","last_change":"2024-09-14T07:01:29.295627+0000","last_active":"2024-09-14T07:01:49.116394+0000","last_peered":"2024-09-14T07:01:49.116394+0000","last_clean":"2024-09-14T07:01:49.116394+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:01:49.116394+0000","last_undegraded":"2024-09-14T07:01:49.116394+0000","last_fullsized":"2024-09-14T07:01:49.116394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:58.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095644100000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3634,"num_read_kb":12358,"num_write":404,"num_write_kb":6155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"130'622","reported_seq":11103,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.238002+0000","last_change":"2024-09-14T07:01:47.492336+0000","last_active":"2024-09-14T07:01:49.238002+0000","last_peered":"2024-09-14T07:01:49.238002+0000","last_clean":"2024-09-14T07:01:49.238002+0000","last_became_active":"2024-09-14T07:01:47.490646+0000","last_became_peered":"2024-09-14T07:01:47.490646+0000","last_unstale":"2024-09-14T07:01:49.238002+0000","last_undegraded":"2024-09-14T07:01:49.238002+0000","last_fullsized":"2024-09-14T07:01:49.238002+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:06:20.079036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001614826,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12269,"num_read_kb":24191,"num_write":663,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"131'725","reported_seq":7151,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237968+0000","last_change":"2024-09-14T07:01:47.487409+0000","last_active":"2024-09-14T07:01:49.237968+0000","last_peered":"2024-09-14T07:01:49.237968+0000","last_clean":"2024-09-14T07:01:49.237968+0000","last_became_active":"2024-09-14T07:01:47.485532+0000","last_became_peered":"2024-09-14T07:01:47.485532+0000","last_unstale":"2024-09-14T07:01:49.237968+0000","last_undegraded":"2024-09-14T07:01:49.237968+0000","last_fullsized":"2024-09-14T07:01:49.237968+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:55.605522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071147799999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8637,"num_read_kb":13164,"num_write":610,"num_write_kb":4987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"131'615","reported_seq":6529,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.488972+0000","last_change":"2024-09-14T07:01:29.296192+0000","last_active":"2024-09-14T07:01:48.488972+0000","last_peered":"2024-09-14T07:01:48.488972+0000","last_clean":"2024-09-14T07:01:48.488972+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:01:48.488972+0000","last_undegraded":"2024-09-14T07:01:48.488972+0000","last_fullsized":"2024-09-14T07:01:48.488972+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:49:30.451851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019616249999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6772,"num_read_kb":10244,"num_write":478,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":579,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":575,"num_read":241540,"num_read_kb":530952,"num_write":22797,"num_write_kb":271683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3188,"num_bytes_recovered":84516668,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20427,"ondisk_log_size":20427,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41276,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330564,"statfs":{"total":96636764160,"available":96594497536,"internally_reserved":0,"allocated":1810432,"data_stored":872232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509476,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59344,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312496,"statfs":{"total":96636764160,"available":96575995904,"internally_reserved":0,"allocated":1822720,"data_stored":878219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59332,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312508,"statfs":{"total":96636764160,"available":96576008192,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64948,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306892,"statfs":{"total":96636764160,"available":96570257408,"internally_reserved":0,"allocated":1867776,"data_stored":1462600,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509475,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69072,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302768,"statfs":{"total":96636764160,"available":96566034432,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1880064,"data_stored":1468587,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45436,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326404,"statfs":{"total":96636764160,"available":96590237696,"internally_reserved":0,"allocated":1806336,"data_stored":872213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:01:53.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T07:01:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:53.997 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"fsid":"32b95aaa-76a2-46ec-8e7f-033ebfa9f5b6","created":"2024-09-14T06:58:59.479055+0000","modified":"2024-09-14T07:01:48.485101+0000","last_up_change":"2024-09-14T06:59:11.718934+0000","last_in_change":"2024-09-14T07:01:32.401522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:59:11.765419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:59:18.343475+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T06:59:27.257805+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":36,"snap_epoch":133,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c036038d-cffb-4584-8851-69604a31ee40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6817","nonce":2081065366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6819","nonce":2081065366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6823","nonce":2081065366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":2081065366},{"type":"v1","addr":"172.21.15.7:6821","nonce":2081065366}]},"public_addr":"172.21.15.7:6817/2081065366","cluster_addr":"172.21.15.7:6819/2081065366","heartbeat_back_addr":"172.21.15.7:6823/2081065366","heartbeat_front_addr":"172.21.15.7:6821/2081065366","state":["exists","up"]},{"osd":1,"uuid":"1c80080f-932a-4957-b64b-f30c4bdb9340","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6806","nonce":1688050909}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6810","nonce":1688050909}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6815","nonce":1688050909}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1688050909},{"type":"v1","addr":"172.21.15.7:6813","nonce":1688050909}]},"public_addr":"172.21.15.7:6806/1688050909","cluster_addr":"172.21.15.7:6810/1688050909","heartbeat_back_addr":"172.21.15.7:6815/1688050909","heartbeat_front_addr":"172.21.15.7:6813/1688050909","state":["exists","up"]},{"osd":2,"uuid":"f3544d11-8a0f-498c-a03c-5d34134c9773","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6825","nonce":1872259061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6827","nonce":1872259061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6831","nonce":1872259061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":1872259061},{"type":"v1","addr":"172.21.15.7:6829","nonce":1872259061}]},"public_addr":"172.21.15.7:6825/1872259061","cluster_addr":"172.21.15.7:6827/1872259061","heartbeat_back_addr":"172.21.15.7:6831/1872259061","heartbeat_front_addr":"172.21.15.7:6829/1872259061","state":["exists","up"]},{"osd":3,"uuid":"d2261455-c91b-414b-925f-388f0e094dbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6801","nonce":1193301030}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6803","nonce":1193301030}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6809","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6811","nonce":1193301030}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6805","nonce":1193301030},{"type":"v1","addr":"172.21.15.7:6807","nonce":1193301030}]},"public_addr":"172.21.15.7:6801/1193301030","cluster_addr":"172.21.15.7:6803/1193301030","heartbeat_back_addr":"172.21.15.7:6811/1193301030","heartbeat_front_addr":"172.21.15.7:6807/1193301030","state":["exists","up"]},{"osd":4,"uuid":"4088d23b-0d06-4415-9eba-6fb64355746c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6811","nonce":2283562461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6813","nonce":2283562461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6817","nonce":2283562461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":2283562461},{"type":"v1","addr":"172.21.15.189:6815","nonce":2283562461}]},"public_addr":"172.21.15.189:6811/2283562461","cluster_addr":"172.21.15.189:6813/2283562461","heartbeat_back_addr":"172.21.15.189:6817/2283562461","heartbeat_front_addr":"172.21.15.189:6815/2283562461","state":["exists","up"]},{"osd":5,"uuid":"5e27cb53-ecce-4061-bc5a-be599bf18cc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6819","nonce":2212769626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6821","nonce":2212769626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6825","nonce":2212769626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":2212769626},{"type":"v1","addr":"172.21.15.189:6823","nonce":2212769626}]},"public_addr":"172.21.15.189:6819/2212769626","cluster_addr":"172.21.15.189:6821/2212769626","heartbeat_back_addr":"172.21.15.189:6825/2212769626","heartbeat_front_addr":"172.21.15.189:6823/2212769626","state":["exists","up"]},{"osd":6,"uuid":"ceb9a1f1-f54c-48d5-91d8-2287cac96ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6827","nonce":3197231389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6829","nonce":3197231389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6833","nonce":3197231389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3197231389},{"type":"v1","addr":"172.21.15.189:6831","nonce":3197231389}]},"public_addr":"172.21.15.189:6827/3197231389","cluster_addr":"172.21.15.189:6829/3197231389","heartbeat_back_addr":"172.21.15.189:6833/3197231389","heartbeat_front_addr":"172.21.15.189:6831/3197231389","state":["exists","up"]},{"osd":7,"uuid":"85c59cd6-6e2b-4cc0-852d-d1f075a8826d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6801","nonce":2162534312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6803","nonce":2162534312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6807","nonce":2162534312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":2162534312},{"type":"v1","addr":"172.21.15.189:6805","nonce":2162534312}]},"public_addr":"172.21.15.189:6801/2162534312","cluster_addr":"172.21.15.189:6803/2162534312","heartbeat_back_addr":"172.21.15.189:6807/2162534312","heartbeat_front_addr":"172.21.15.189:6805/2162534312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.064941+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.780598+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.127700+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.730995+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.864423+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:09.329361+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:10.107581+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:59:08.300043+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.8","osds":[7,0]},{"pgid":"3.9","osds":[4,6]},{"pgid":"3.11","osds":[6,7]},{"pgid":"3.18","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":7,"to":5}]},{"pgid":"3.1f","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T07:01:55.011 INFO:tasks.ceph:Scrubbing osd.0 2024-09-14T07:01:55.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:01:55.153 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:55.154 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:01:55.154 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:55.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-14T07:01:55.465 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-09-14T07:01:55.478 INFO:tasks.ceph:Scrubbing osd.1 2024-09-14T07:01:55.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:01:55.607 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:55.607 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:01:55.607 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:55.618 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-14T07:01:55.917 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-09-14T07:01:55.929 INFO:tasks.ceph:Scrubbing osd.2 2024-09-14T07:01:55.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:01:56.057 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:56.057 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:01:56.057 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:56.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-14T07:01:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-09-14T07:01:56.381 INFO:tasks.ceph:Scrubbing osd.3 2024-09-14T07:01:56.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:01:56.508 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:56.509 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:01:56.509 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:56.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-14T07:01:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-09-14T07:01:56.834 INFO:tasks.ceph:Scrubbing osd.4 2024-09-14T07:01:56.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:01:56.963 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:56.963 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:01:56.963 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:56.974 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-14T07:01:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-09-14T07:01:57.294 INFO:tasks.ceph:Scrubbing osd.5 2024-09-14T07:01:57.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:01:57.421 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:57.422 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:01:57.422 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:57.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-14T07:01:57.732 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-09-14T07:01:57.748 INFO:tasks.ceph:Scrubbing osd.6 2024-09-14T07:01:57.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:01:57.876 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:57.876 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:01:57.876 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:57.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-14T07:01:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2024-09-14T07:01:58.200 INFO:tasks.ceph:Scrubbing osd.7 2024-09-14T07:01:58.201 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-14T07:01:58.335 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-14T07:01:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T07:01:58.335 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-14T07:01:58.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-14T07:01:58.650 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2024-09-14T07:01:58.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:01:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:01:58.967 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:01:58.979 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-14T07:01:57.779264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":575,"num_read":241606,"num_read_kb":531007,"num_write":22900,"num_write_kb":273313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3193,"num_bytes_recovered":84516725,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20564,"ondisk_log_size":20564,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":431804,"kb_used_data":14336,"kb_used_omap":12,"kb_used_meta":417267,"kb_avail":754542916,"statfs":{"total":773094113280,"available":772651945984,"internally_reserved":0,"allocated":14680064,"data_stored":8206412,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":427281969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002495"},"pg_stats":[{"pgid":"3.1f","version":"133'538","reported_seq":7941,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489581+0000","last_change":"2024-09-14T07:01:29.346010+0000","last_active":"2024-09-14T07:01:48.489581+0000","last_peered":"2024-09-14T07:01:48.489581+0000","last_clean":"2024-09-14T07:01:48.489581+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:01:48.489581+0000","last_undegraded":"2024-09-14T07:01:48.489581+0000","last_fullsized":"2024-09-14T07:01:48.489581+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:00.019940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099215999999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8424,"num_read_kb":37327,"num_write":508,"num_write_kb":21659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"130'423","reported_seq":3021,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489455+0000","last_change":"2024-09-14T07:01:29.367999+0000","last_active":"2024-09-14T07:01:48.489455+0000","last_peered":"2024-09-14T07:01:48.489455+0000","last_clean":"2024-09-14T07:01:48.489455+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:01:48.489455+0000","last_undegraded":"2024-09-14T07:01:48.489455+0000","last_fullsized":"2024-09-14T07:01:48.489455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:38.115466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001057501,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2716,"num_read_kb":8402,"num_write":360,"num_write_kb":7998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"130'339","reported_seq":5823,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:43.465257+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T07:01:43.462852+0000","last_became_peered":"2024-09-14T07:01:43.462852+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":339,"log_dups_size":0,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:42:36.409964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000797209,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5182,"num_read_kb":12861,"num_write":295,"num_write_kb":5166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"133'2042","reported_seq":4848,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116619+0000","last_change":"2024-09-14T07:01:29.296724+0000","last_active":"2024-09-14T07:01:49.116619+0000","last_peered":"2024-09-14T07:01:49.116619+0000","last_clean":"2024-09-14T07:01:49.116619+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:01:49.116619+0000","last_undegraded":"2024-09-14T07:01:49.116619+0000","last_fullsized":"2024-09-14T07:01:49.116619+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":2042,"log_dups_size":0,"ondisk_log_size":2042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:44.108614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096933700000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11671,"num_read_kb":22543,"num_write":3453,"num_write_kb":11219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"131'900","reported_seq":3379,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116967+0000","last_change":"2024-09-14T07:01:29.299798+0000","last_active":"2024-09-14T07:01:49.116967+0000","last_peered":"2024-09-14T07:01:49.116967+0000","last_clean":"2024-09-14T07:01:49.116967+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:01:49.116967+0000","last_undegraded":"2024-09-14T07:01:49.116967+0000","last_fullsized":"2024-09-14T07:01:49.116967+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:51.597368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087443700000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4436,"num_read_kb":12231,"num_write":1137,"num_write_kb":20066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"133'1187","reported_seq":8951,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116584+0000","last_change":"2024-09-14T07:01:29.296969+0000","last_active":"2024-09-14T07:01:49.116584+0000","last_peered":"2024-09-14T07:01:49.116584+0000","last_clean":"2024-09-14T07:01:49.116584+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:01:49.116584+0000","last_undegraded":"2024-09-14T07:01:49.116584+0000","last_fullsized":"2024-09-14T07:01:49.116584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:26:48.784038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015092009999999999,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":13058,"num_read_kb":21088,"num_write":1639,"num_write_kb":6895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"133'517","reported_seq":5016,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.117019+0000","last_change":"2024-09-14T07:01:29.300460+0000","last_active":"2024-09-14T07:01:49.117019+0000","last_peered":"2024-09-14T07:01:49.117019+0000","last_clean":"2024-09-14T07:01:49.117019+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:01:49.117019+0000","last_undegraded":"2024-09-14T07:01:49.117019+0000","last_fullsized":"2024-09-14T07:01:49.117019+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:54:21.579346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00137473,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4742,"num_read_kb":12493,"num_write":520,"num_write_kb":11830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"132'662","reported_seq":3807,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489245+0000","last_change":"2024-09-14T07:01:29.346100+0000","last_active":"2024-09-14T07:01:48.489245+0000","last_peered":"2024-09-14T07:01:48.489245+0000","last_clean":"2024-09-14T07:01:48.489245+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:01:48.489245+0000","last_undegraded":"2024-09-14T07:01:48.489245+0000","last_fullsized":"2024-09-14T07:01:48.489245+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:15.330383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011507799999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5024,"num_read_kb":10270,"num_write":682,"num_write_kb":6392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"130'392","reported_seq":6848,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489564+0000","last_change":"2024-09-14T07:01:29.368091+0000","last_active":"2024-09-14T07:01:48.489564+0000","last_peered":"2024-09-14T07:01:48.489564+0000","last_clean":"2024-09-14T07:01:48.489564+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:01:48.489564+0000","last_undegraded":"2024-09-14T07:01:48.489564+0000","last_fullsized":"2024-09-14T07:01:48.489564+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:17.259452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010952469999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7606,"num_read_kb":16770,"num_write":322,"num_write_kb":2846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"129'413","reported_seq":5194,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116357+0000","last_change":"2024-09-14T07:01:43.464245+0000","last_active":"2024-09-14T07:01:49.116357+0000","last_peered":"2024-09-14T07:01:49.116357+0000","last_clean":"2024-09-14T07:01:49.116357+0000","last_became_active":"2024-09-14T07:01:43.462372+0000","last_became_peered":"2024-09-14T07:01:43.462372+0000","last_unstale":"2024-09-14T07:01:49.116357+0000","last_undegraded":"2024-09-14T07:01:49.116357+0000","last_fullsized":"2024-09-14T07:01:49.116357+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:32:17.252003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022563269999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4902,"num_read_kb":8441,"num_write":353,"num_write_kb":3835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"130'346","reported_seq":2784,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489607+0000","last_change":"2024-09-14T07:01:36.306565+0000","last_active":"2024-09-14T07:01:48.489607+0000","last_peered":"2024-09-14T07:01:48.489607+0000","last_clean":"2024-09-14T07:01:48.489607+0000","last_became_active":"2024-09-14T07:01:36.304931+0000","last_became_peered":"2024-09-14T07:01:36.304931+0000","last_unstale":"2024-09-14T07:01:48.489607+0000","last_undegraded":"2024-09-14T07:01:48.489607+0000","last_fullsized":"2024-09-14T07:01:48.489607+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020075470000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1776,"num_read_kb":3899,"num_write":305,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"131'789","reported_seq":5674,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489195+0000","last_change":"2024-09-14T07:01:43.465640+0000","last_active":"2024-09-14T07:01:48.489195+0000","last_peered":"2024-09-14T07:01:48.489195+0000","last_clean":"2024-09-14T07:01:48.489195+0000","last_became_active":"2024-09-14T07:01:43.461464+0000","last_became_peered":"2024-09-14T07:01:43.461464+0000","last_unstale":"2024-09-14T07:01:48.489195+0000","last_undegraded":"2024-09-14T07:01:48.489195+0000","last_fullsized":"2024-09-14T07:01:48.489195+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":789,"log_dups_size":0,"ondisk_log_size":789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:07.293229+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0087910980000000007,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6730,"num_read_kb":12616,"num_write":1057,"num_write_kb":7169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489294+0000","last_change":"2024-09-14T06:59:21.767340+0000","last_active":"2024-09-14T07:01:48.489294+0000","last_peered":"2024-09-14T07:01:48.489294+0000","last_clean":"2024-09-14T07:01:48.489294+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:01:48.489294+0000","last_undegraded":"2024-09-14T07:01:48.489294+0000","last_fullsized":"2024-09-14T07:01:48.489294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:18.993246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"133'800","reported_seq":7023,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116939+0000","last_change":"2024-09-14T07:01:29.300111+0000","last_active":"2024-09-14T07:01:49.116939+0000","last_peered":"2024-09-14T07:01:49.116939+0000","last_clean":"2024-09-14T07:01:49.116939+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:01:49.116939+0000","last_undegraded":"2024-09-14T07:01:49.116939+0000","last_fullsized":"2024-09-14T07:01:49.116939+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:46.258973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010359880000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8046,"num_read_kb":9156,"num_write":887,"num_write_kb":5801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.117092+0000","last_change":"2024-09-14T06:59:21.902281+0000","last_active":"2024-09-14T07:01:49.117092+0000","last_peered":"2024-09-14T07:01:49.117092+0000","last_clean":"2024-09-14T07:01:49.117092+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:01:49.117092+0000","last_undegraded":"2024-09-14T07:01:49.117092+0000","last_fullsized":"2024-09-14T07:01:49.117092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.486370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"132'510","reported_seq":11748,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116991+0000","last_change":"2024-09-14T07:01:36.306671+0000","last_active":"2024-09-14T07:01:49.116991+0000","last_peered":"2024-09-14T07:01:49.116991+0000","last_clean":"2024-09-14T07:01:49.116991+0000","last_became_active":"2024-09-14T07:01:36.305222+0000","last_became_peered":"2024-09-14T07:01:36.305222+0000","last_unstale":"2024-09-14T07:01:49.116991+0000","last_undegraded":"2024-09-14T07:01:49.116991+0000","last_fullsized":"2024-09-14T07:01:49.116991+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":510,"log_dups_size":0,"ondisk_log_size":510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:26:12.004745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001033576,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12680,"num_read_kb":20099,"num_write":483,"num_write_kb":7950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":281,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489378+0000","last_change":"2024-09-14T06:59:21.766844+0000","last_active":"2024-09-14T07:01:48.489378+0000","last_peered":"2024-09-14T07:01:48.489378+0000","last_clean":"2024-09-14T07:01:48.489378+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:01:48.489378+0000","last_undegraded":"2024-09-14T07:01:48.489378+0000","last_fullsized":"2024-09-14T07:01:48.489378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:35.226236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"129'508","reported_seq":3947,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237958+0000","last_change":"2024-09-14T07:01:47.492455+0000","last_active":"2024-09-14T07:01:49.237958+0000","last_peered":"2024-09-14T07:01:49.237958+0000","last_clean":"2024-09-14T07:01:49.237958+0000","last_became_active":"2024-09-14T07:01:47.490786+0000","last_became_peered":"2024-09-14T07:01:47.490786+0000","last_unstale":"2024-09-14T07:01:49.237958+0000","last_undegraded":"2024-09-14T07:01:49.237958+0000","last_fullsized":"2024-09-14T07:01:49.237958+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:11.019913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070556600000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4047,"num_read_kb":4338,"num_write":428,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":305,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489570+0000","last_change":"2024-09-14T07:01:33.416728+0000","last_active":"2024-09-14T07:01:48.489570+0000","last_peered":"2024-09-14T07:01:48.489570+0000","last_clean":"2024-09-14T07:01:48.489570+0000","last_became_active":"2024-09-14T07:01:33.416397+0000","last_became_peered":"2024-09-14T07:01:33.416397+0000","last_unstale":"2024-09-14T07:01:48.489570+0000","last_undegraded":"2024-09-14T07:01:48.489570+0000","last_fullsized":"2024-09-14T07:01:48.489570+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:43:56.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"152'592","reported_seq":10491,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:57.233424+0000","last_change":"2024-09-14T07:01:57.233424+0000","last_active":"2024-09-14T07:01:57.233424+0000","last_peered":"2024-09-14T07:01:57.233424+0000","last_clean":"2024-09-14T07:01:57.233424+0000","last_became_active":"2024-09-14T07:01:36.302844+0000","last_became_peered":"2024-09-14T07:01:36.302844+0000","last_unstale":"2024-09-14T07:01:57.233424+0000","last_undegraded":"2024-09-14T07:01:57.233424+0000","last_fullsized":"2024-09-14T07:01:57.233424+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'592","last_scrub_stamp":"2024-09-14T07:01:57.233313+0000","last_deep_scrub":"152'592","last_deep_scrub_stamp":"2024-09-14T07:01:57.233313+0000","last_clean_scrub_stamp":"2024-09-14T07:01:57.233313+0000","objects_scrubbed":19,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:35.167493+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00093736199999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8151,"num_read_kb":12914,"num_write":603,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"133'814","reported_seq":8542,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489516+0000","last_change":"2024-09-14T07:01:35.302802+0000","last_active":"2024-09-14T07:01:48.489516+0000","last_peered":"2024-09-14T07:01:48.489516+0000","last_clean":"2024-09-14T07:01:48.489516+0000","last_became_active":"2024-09-14T07:01:35.301421+0000","last_became_peered":"2024-09-14T07:01:35.301421+0000","last_unstale":"2024-09-14T07:01:48.489516+0000","last_undegraded":"2024-09-14T07:01:48.489516+0000","last_fullsized":"2024-09-14T07:01:48.489516+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":814,"log_dups_size":0,"ondisk_log_size":814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:41:57.632029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027805429999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":8837,"num_read_kb":11133,"num_write":1016,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":31,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489044+0000","last_change":"2024-09-14T07:01:40.447790+0000","last_active":"2024-09-14T07:01:48.489044+0000","last_peered":"2024-09-14T07:01:48.489044+0000","last_clean":"2024-09-14T07:01:48.489044+0000","last_became_active":"2024-09-14T07:01:40.444161+0000","last_became_peered":"2024-09-14T07:01:40.444161+0000","last_unstale":"2024-09-14T07:01:48.489044+0000","last_undegraded":"2024-09-14T07:01:48.489044+0000","last_fullsized":"2024-09-14T07:01:48.489044+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:38:14.614944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"131'477","reported_seq":22200,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489342+0000","last_change":"2024-09-14T07:01:29.368340+0000","last_active":"2024-09-14T07:01:48.489342+0000","last_peered":"2024-09-14T07:01:48.489342+0000","last_clean":"2024-09-14T07:01:48.489342+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:01:48.489342+0000","last_undegraded":"2024-09-14T07:01:48.489342+0000","last_fullsized":"2024-09-14T07:01:48.489342+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:44.687681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089711299999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":22342,"num_read_kb":44381,"num_write":481,"num_write_kb":17365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":280,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116984+0000","last_change":"2024-09-14T06:59:21.902506+0000","last_active":"2024-09-14T07:01:49.116984+0000","last_peered":"2024-09-14T07:01:49.116984+0000","last_clean":"2024-09-14T07:01:49.116984+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:01:49.116984+0000","last_undegraded":"2024-09-14T07:01:49.116984+0000","last_fullsized":"2024-09-14T07:01:49.116984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:06:28.283405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"129'389","reported_seq":2578,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237880+0000","last_change":"2024-09-14T07:01:47.487342+0000","last_active":"2024-09-14T07:01:49.237880+0000","last_peered":"2024-09-14T07:01:49.237880+0000","last_clean":"2024-09-14T07:01:49.237880+0000","last_became_active":"2024-09-14T07:01:47.485661+0000","last_became_peered":"2024-09-14T07:01:47.485661+0000","last_unstale":"2024-09-14T07:01:49.237880+0000","last_undegraded":"2024-09-14T07:01:49.237880+0000","last_fullsized":"2024-09-14T07:01:49.237880+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:10.350553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001602144,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3106,"num_read_kb":12645,"num_write":311,"num_write_kb":7126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":289,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:57.083994+0000","last_change":"2024-09-14T07:01:57.083994+0000","last_active":"2024-09-14T07:01:57.083994+0000","last_peered":"2024-09-14T07:01:57.083994+0000","last_clean":"2024-09-14T07:01:57.083994+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:01:57.083994+0000","last_undegraded":"2024-09-14T07:01:57.083994+0000","last_fullsized":"2024-09-14T07:01:57.083994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:01:57.083893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:01:57.083893+0000","last_clean_scrub_stamp":"2024-09-14T07:01:57.083893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:03:55.085531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"132'710","reported_seq":10681,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116913+0000","last_change":"2024-09-14T07:01:29.299674+0000","last_active":"2024-09-14T07:01:49.116913+0000","last_peered":"2024-09-14T07:01:49.116913+0000","last_clean":"2024-09-14T07:01:49.116913+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:01:49.116913+0000","last_undegraded":"2024-09-14T07:01:49.116913+0000","last_fullsized":"2024-09-14T07:01:49.116913+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:09.671718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074696400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11440,"num_read_kb":36601,"num_write":924,"num_write_kb":12561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":302,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489125+0000","last_change":"2024-09-14T07:00:27.523385+0000","last_active":"2024-09-14T07:01:48.489125+0000","last_peered":"2024-09-14T07:01:48.489125+0000","last_clean":"2024-09-14T07:01:48.489125+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:01:48.489125+0000","last_undegraded":"2024-09-14T07:01:48.489125+0000","last_fullsized":"2024-09-14T07:01:48.489125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:18.742248+0000","last_clean_scrub_stamp":"2024-09-14T06:59:18.742248+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:27:19.504174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"152'74","reported_seq":381,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:56.703019+0000","last_change":"2024-09-14T07:01:56.703019+0000","last_active":"2024-09-14T07:01:56.703019+0000","last_peered":"2024-09-14T07:01:56.703019+0000","last_clean":"2024-09-14T07:01:56.703019+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:01:56.703019+0000","last_undegraded":"2024-09-14T07:01:56.703019+0000","last_fullsized":"2024-09-14T07:01:56.703019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'74","last_scrub_stamp":"2024-09-14T07:01:56.702943+0000","last_deep_scrub":"152'74","last_deep_scrub_stamp":"2024-09-14T07:01:56.702943+0000","last_clean_scrub_stamp":"2024-09-14T07:01:56.702943+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:43:12.611623+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"130'594","reported_seq":10088,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116783+0000","last_change":"2024-09-14T07:01:29.297332+0000","last_active":"2024-09-14T07:01:49.116783+0000","last_peered":"2024-09-14T07:01:49.116783+0000","last_clean":"2024-09-14T07:01:49.116783+0000","last_became_active":"2024-09-14T07:00:53.610102+0000","last_became_peered":"2024-09-14T07:00:53.610102+0000","last_unstale":"2024-09-14T07:01:49.116783+0000","last_undegraded":"2024-09-14T07:01:49.116783+0000","last_fullsized":"2024-09-14T07:01:49.116783+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:58.662454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097674599999999991,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":9511,"num_read_kb":20954,"num_write":534,"num_write_kb":10486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4712244,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"133'769","reported_seq":6051,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116875+0000","last_change":"2024-09-14T07:01:37.305171+0000","last_active":"2024-09-14T07:01:49.116875+0000","last_peered":"2024-09-14T07:01:49.116875+0000","last_clean":"2024-09-14T07:01:49.116875+0000","last_became_active":"2024-09-14T07:01:37.303369+0000","last_became_peered":"2024-09-14T07:01:37.303369+0000","last_unstale":"2024-09-14T07:01:49.116875+0000","last_undegraded":"2024-09-14T07:01:49.116875+0000","last_fullsized":"2024-09-14T07:01:49.116875+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:52.195099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024576519999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8449,"num_read_kb":18168,"num_write":805,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"132'1123","reported_seq":8414,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489096+0000","last_change":"2024-09-14T07:01:29.295466+0000","last_active":"2024-09-14T07:01:48.489096+0000","last_peered":"2024-09-14T07:01:48.489096+0000","last_clean":"2024-09-14T07:01:48.489096+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:01:48.489096+0000","last_undegraded":"2024-09-14T07:01:48.489096+0000","last_fullsized":"2024-09-14T07:01:48.489096+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:23:01.075768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00157712,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":10507,"num_read_kb":9740,"num_write":1539,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"131'352","reported_seq":3167,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489439+0000","last_change":"2024-09-14T07:01:36.307001+0000","last_active":"2024-09-14T07:01:48.489439+0000","last_peered":"2024-09-14T07:01:48.489439+0000","last_clean":"2024-09-14T07:01:48.489439+0000","last_became_active":"2024-09-14T07:01:36.305298+0000","last_became_peered":"2024-09-14T07:01:36.305298+0000","last_unstale":"2024-09-14T07:01:48.489439+0000","last_undegraded":"2024-09-14T07:01:48.489439+0000","last_fullsized":"2024-09-14T07:01:48.489439+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:38:17.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074805599999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1936,"num_read_kb":8361,"num_write":285,"num_write_kb":3165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"152'473","reported_seq":5833,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:56.207960+0000","last_change":"2024-09-14T07:01:56.207960+0000","last_active":"2024-09-14T07:01:56.207960+0000","last_peered":"2024-09-14T07:01:56.207960+0000","last_clean":"2024-09-14T07:01:56.207960+0000","last_became_active":"2024-09-14T07:01:35.302427+0000","last_became_peered":"2024-09-14T07:01:35.302427+0000","last_unstale":"2024-09-14T07:01:56.207960+0000","last_undegraded":"2024-09-14T07:01:56.207960+0000","last_fullsized":"2024-09-14T07:01:56.207960+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'473","last_scrub_stamp":"2024-09-14T07:01:56.207878+0000","last_deep_scrub":"152'473","last_deep_scrub_stamp":"2024-09-14T07:01:56.207878+0000","last_clean_scrub_stamp":"2024-09-14T07:01:56.207878+0000","objects_scrubbed":13,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:24:15.071245+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.000724418,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5156,"num_read_kb":11268,"num_write":417,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"133'441","reported_seq":2271,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489369+0000","last_change":"2024-09-14T07:01:29.295502+0000","last_active":"2024-09-14T07:01:48.489369+0000","last_peered":"2024-09-14T07:01:48.489369+0000","last_clean":"2024-09-14T07:01:48.489369+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:01:48.489369+0000","last_undegraded":"2024-09-14T07:01:48.489369+0000","last_fullsized":"2024-09-14T07:01:48.489369+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:12.777971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015159500000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1920,"num_read_kb":6604,"num_write":451,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"132'450","reported_seq":12399,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116494+0000","last_change":"2024-09-14T07:01:29.344029+0000","last_active":"2024-09-14T07:01:49.116494+0000","last_peered":"2024-09-14T07:01:49.116494+0000","last_clean":"2024-09-14T07:01:49.116494+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:01:49.116494+0000","last_undegraded":"2024-09-14T07:01:49.116494+0000","last_fullsized":"2024-09-14T07:01:49.116494+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:57.081265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001728802,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":12086,"num_read_kb":54262,"num_write":423,"num_write_kb":18409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"152'535","reported_seq":4896,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:57.187807+0000","last_change":"2024-09-14T07:01:57.187807+0000","last_active":"2024-09-14T07:01:57.187807+0000","last_peered":"2024-09-14T07:01:57.187807+0000","last_clean":"2024-09-14T07:01:57.187807+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:01:57.187807+0000","last_undegraded":"2024-09-14T07:01:57.187807+0000","last_fullsized":"2024-09-14T07:01:57.187807+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'535","last_scrub_stamp":"2024-09-14T07:01:57.187738+0000","last_deep_scrub":"152'535","last_deep_scrub_stamp":"2024-09-14T07:01:57.187738+0000","last_clean_scrub_stamp":"2024-09-14T07:01:57.187738+0000","objects_scrubbed":28,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:34:40.774916+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.001321022,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5747,"num_read_kb":11430,"num_write":424,"num_write_kb":4665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"131'440","reported_seq":4097,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.116394+0000","last_change":"2024-09-14T07:01:29.295627+0000","last_active":"2024-09-14T07:01:49.116394+0000","last_peered":"2024-09-14T07:01:49.116394+0000","last_clean":"2024-09-14T07:01:49.116394+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:01:49.116394+0000","last_undegraded":"2024-09-14T07:01:49.116394+0000","last_fullsized":"2024-09-14T07:01:49.116394+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:58.183679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095644100000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3634,"num_read_kb":12358,"num_write":404,"num_write_kb":6155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"130'622","reported_seq":11103,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.238002+0000","last_change":"2024-09-14T07:01:47.492336+0000","last_active":"2024-09-14T07:01:49.238002+0000","last_peered":"2024-09-14T07:01:49.238002+0000","last_clean":"2024-09-14T07:01:49.238002+0000","last_became_active":"2024-09-14T07:01:47.490646+0000","last_became_peered":"2024-09-14T07:01:47.490646+0000","last_unstale":"2024-09-14T07:01:49.238002+0000","last_undegraded":"2024-09-14T07:01:49.238002+0000","last_fullsized":"2024-09-14T07:01:49.238002+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:06:20.079036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001614826,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12269,"num_read_kb":24191,"num_write":663,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"131'725","reported_seq":7151,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:49.237968+0000","last_change":"2024-09-14T07:01:47.487409+0000","last_active":"2024-09-14T07:01:49.237968+0000","last_peered":"2024-09-14T07:01:49.237968+0000","last_clean":"2024-09-14T07:01:49.237968+0000","last_became_active":"2024-09-14T07:01:47.485532+0000","last_became_peered":"2024-09-14T07:01:47.485532+0000","last_unstale":"2024-09-14T07:01:49.237968+0000","last_undegraded":"2024-09-14T07:01:49.237968+0000","last_fullsized":"2024-09-14T07:01:49.237968+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:12:55.605522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071147799999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8637,"num_read_kb":13164,"num_write":610,"num_write_kb":4987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"131'615","reported_seq":6530,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:48.489029+0000","last_change":"2024-09-14T07:01:29.296192+0000","last_active":"2024-09-14T07:01:48.489029+0000","last_peered":"2024-09-14T07:01:48.489029+0000","last_clean":"2024-09-14T07:01:48.489029+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:01:48.489029+0000","last_undegraded":"2024-09-14T07:01:48.489029+0000","last_fullsized":"2024-09-14T07:01:48.489029+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:59:27.785355+0000","last_clean_scrub_stamp":"2024-09-14T06:59:27.785355+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:49:30.451851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019616249999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6772,"num_read_kb":10244,"num_write":478,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":579,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":575,"num_read":241540,"num_read_kb":530952,"num_write":22797,"num_write_kb":271683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3188,"num_bytes_recovered":84516668,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20487,"ondisk_log_size":20487,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509478,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41292,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330548,"statfs":{"total":96636764160,"available":96594481152,"internally_reserved":0,"allocated":1822720,"data_stored":878219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509478,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59344,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312496,"statfs":{"total":96636764160,"available":96575995904,"internally_reserved":0,"allocated":1822720,"data_stored":878219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509477,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59332,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312508,"statfs":{"total":96636764160,"available":96576008192,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509477,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64964,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306876,"statfs":{"total":96636764160,"available":96570241024,"internally_reserved":0,"allocated":1880064,"data_stored":1468587,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509477,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69056,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302784,"statfs":{"total":96636764160,"available":96566050816,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":1880064,"data_stored":1468587,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55176,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316664,"statfs":{"total":96636764160,"available":96580263936,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45444,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326396,"statfs":{"total":96636764160,"available":96590229504,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:01:58.984 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.984 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.984 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.984 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.985 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.985 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.985 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.985 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.985 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.985 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.985 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.985 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.986 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-14T06:59:18.742248+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=18, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.986 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.986 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-14T06:59:18.742248+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=18, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.986 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.986 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-14T06:59:18.742248+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=18, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.986 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.986 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-14T06:59:18.742248+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=18, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.986 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.987 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-14T06:59:18.742248+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=18, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.987 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.987 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-14T06:59:18.742248+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=18, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.987 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.987 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.987 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-14T06:59:18.742248+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=18, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.987 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.987 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.987 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.988 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.988 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.988 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.988 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.988 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.988 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.988 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-14T06:59:27.785355+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=59, tm_sec=27, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=7, tm_min=1, tm_sec=54, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T07:01:58.988 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-14T07:02:18.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T07:02:19.301 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-14T07:02:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-14T07:02:19.313 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-14T07:02:17.783922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":583,"num_object_clones":0,"num_object_copies":1166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":575,"num_read":241606,"num_read_kb":531007,"num_write":22900,"num_write_kb":273313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3193,"num_bytes_recovered":84516725,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21084,"ondisk_log_size":21084,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":435904,"kb_used_data":14336,"kb_used_omap":12,"kb_used_meta":421427,"kb_avail":754538816,"statfs":{"total":773094113280,"available":772647747584,"internally_reserved":0,"allocated":14680064,"data_stored":8206412,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":431541809},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002697"},"pg_stats":[{"pgid":"3.1f","version":"152'553","reported_seq":7965,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:01.320614+0000","last_change":"2024-09-14T07:02:01.320545+0000","last_active":"2024-09-14T07:02:01.320614+0000","last_peered":"2024-09-14T07:02:01.320614+0000","last_clean":"2024-09-14T07:02:01.320614+0000","last_became_active":"2024-09-14T07:00:01.384826+0000","last_became_peered":"2024-09-14T07:00:01.384826+0000","last_unstale":"2024-09-14T07:02:01.320614+0000","last_undegraded":"2024-09-14T07:02:01.320614+0000","last_fullsized":"2024-09-14T07:02:01.320614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'553","last_scrub_stamp":"2024-09-14T07:02:01.320439+0000","last_deep_scrub":"152'553","last_deep_scrub_stamp":"2024-09-14T07:02:01.320439+0000","last_clean_scrub_stamp":"2024-09-14T07:02:01.320439+0000","objects_scrubbed":15,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:48:06.798630+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00099215999999999996,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":8424,"num_read_kb":37327,"num_write":508,"num_write_kb":21659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4395185,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"152'445","reported_seq":3051,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:01.627592+0000","last_change":"2024-09-14T07:02:01.627592+0000","last_active":"2024-09-14T07:02:01.627592+0000","last_peered":"2024-09-14T07:02:01.627592+0000","last_clean":"2024-09-14T07:02:01.627592+0000","last_became_active":"2024-09-14T07:00:30.291075+0000","last_became_peered":"2024-09-14T07:00:30.291075+0000","last_unstale":"2024-09-14T07:02:01.627592+0000","last_undegraded":"2024-09-14T07:02:01.627592+0000","last_fullsized":"2024-09-14T07:02:01.627592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'445","last_scrub_stamp":"2024-09-14T07:02:01.627453+0000","last_deep_scrub":"152'445","last_deep_scrub_stamp":"2024-09-14T07:02:01.627453+0000","last_clean_scrub_stamp":"2024-09-14T07:02:01.627453+0000","objects_scrubbed":22,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:20.573095+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.001057501,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":2716,"num_read_kb":8402,"num_write":360,"num_write_kb":7998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":4498186,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"152'362","reported_seq":5854,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:04.979134+0000","last_change":"2024-09-14T07:02:04.979134+0000","last_active":"2024-09-14T07:02:04.979134+0000","last_peered":"2024-09-14T07:02:04.979134+0000","last_clean":"2024-09-14T07:02:04.979134+0000","last_became_active":"2024-09-14T07:01:43.462852+0000","last_became_peered":"2024-09-14T07:01:43.462852+0000","last_unstale":"2024-09-14T07:02:04.979134+0000","last_undegraded":"2024-09-14T07:02:04.979134+0000","last_fullsized":"2024-09-14T07:02:04.979134+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'362","last_scrub_stamp":"2024-09-14T07:02:04.979024+0000","last_deep_scrub":"152'362","last_deep_scrub_stamp":"2024-09-14T07:02:04.979024+0000","last_clean_scrub_stamp":"2024-09-14T07:02:04.979024+0000","objects_scrubbed":23,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:43:50.011849+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.000797209,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5182,"num_read_kb":12861,"num_write":295,"num_write_kb":5166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":4113587,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"152'2059","reported_seq":4874,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:59.812888+0000","last_change":"2024-09-14T07:01:59.812813+0000","last_active":"2024-09-14T07:01:59.812888+0000","last_peered":"2024-09-14T07:01:59.812888+0000","last_clean":"2024-09-14T07:01:59.812888+0000","last_became_active":"2024-09-14T06:59:28.801112+0000","last_became_peered":"2024-09-14T06:59:28.801112+0000","last_unstale":"2024-09-14T07:01:59.812888+0000","last_undegraded":"2024-09-14T07:01:59.812888+0000","last_fullsized":"2024-09-14T07:01:59.812888+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'2059","last_scrub_stamp":"2024-09-14T07:01:59.812703+0000","last_deep_scrub":"152'2059","last_deep_scrub_stamp":"2024-09-14T07:01:59.812703+0000","last_clean_scrub_stamp":"2024-09-14T07:01:59.812703+0000","objects_scrubbed":17,"log_size":2059,"log_dups_size":0,"ondisk_log_size":2059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:31.628533+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.00096933700000000002,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":11671,"num_read_kb":22543,"num_write":3453,"num_write_kb":11219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"152'918","reported_seq":3406,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:02.504153+0000","last_change":"2024-09-14T07:02:02.504153+0000","last_active":"2024-09-14T07:02:02.504153+0000","last_peered":"2024-09-14T07:02:02.504153+0000","last_clean":"2024-09-14T07:02:02.504153+0000","last_became_active":"2024-09-14T06:59:28.801624+0000","last_became_peered":"2024-09-14T06:59:28.801624+0000","last_unstale":"2024-09-14T07:02:02.504153+0000","last_undegraded":"2024-09-14T07:02:02.504153+0000","last_fullsized":"2024-09-14T07:02:02.504153+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'918","last_scrub_stamp":"2024-09-14T07:02:02.504061+0000","last_deep_scrub":"152'918","last_deep_scrub_stamp":"2024-09-14T07:02:02.504061+0000","last_clean_scrub_stamp":"2024-09-14T07:02:02.504061+0000","objects_scrubbed":18,"log_size":918,"log_dups_size":0,"ondisk_log_size":918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:20:54.287100+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.00087443700000000004,"stat_sum":{"num_bytes":0,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":18,"num_read":4436,"num_read_kb":12231,"num_write":1137,"num_write_kb":20066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"152'1207","reported_seq":8980,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:58.760092+0000","last_change":"2024-09-14T07:01:58.759987+0000","last_active":"2024-09-14T07:01:58.760092+0000","last_peered":"2024-09-14T07:01:58.760092+0000","last_clean":"2024-09-14T07:01:58.760092+0000","last_became_active":"2024-09-14T06:59:28.805153+0000","last_became_peered":"2024-09-14T06:59:28.805153+0000","last_unstale":"2024-09-14T07:01:58.760092+0000","last_undegraded":"2024-09-14T07:01:58.760092+0000","last_fullsized":"2024-09-14T07:01:58.760092+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'1207","last_scrub_stamp":"2024-09-14T07:01:58.759844+0000","last_deep_scrub":"152'1207","last_deep_scrub_stamp":"2024-09-14T07:01:58.759844+0000","last_clean_scrub_stamp":"2024-09-14T07:01:58.759844+0000","objects_scrubbed":20,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:00:49.801152+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0015092009999999999,"stat_sum":{"num_bytes":19,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":19,"num_read":13058,"num_read_kb":21088,"num_write":1639,"num_write_kb":6895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"152'533","reported_seq":5040,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:03.521232+0000","last_change":"2024-09-14T07:02:03.521232+0000","last_active":"2024-09-14T07:02:03.521232+0000","last_peered":"2024-09-14T07:02:03.521232+0000","last_clean":"2024-09-14T07:02:03.521232+0000","last_became_active":"2024-09-14T06:59:28.803508+0000","last_became_peered":"2024-09-14T06:59:28.803508+0000","last_unstale":"2024-09-14T07:02:03.521232+0000","last_undegraded":"2024-09-14T07:02:03.521232+0000","last_fullsized":"2024-09-14T07:02:03.521232+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'533","last_scrub_stamp":"2024-09-14T07:02:03.521107+0000","last_deep_scrub":"152'533","last_deep_scrub_stamp":"2024-09-14T07:02:03.521107+0000","last_clean_scrub_stamp":"2024-09-14T07:02:03.521107+0000","objects_scrubbed":16,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:31:08.816697+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00137473,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":4742,"num_read_kb":12493,"num_write":520,"num_write_kb":11830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.18","version":"152'679","reported_seq":3833,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:00.271596+0000","last_change":"2024-09-14T07:02:00.271523+0000","last_active":"2024-09-14T07:02:00.271596+0000","last_peered":"2024-09-14T07:02:00.271596+0000","last_clean":"2024-09-14T07:02:00.271596+0000","last_became_active":"2024-09-14T06:59:37.044304+0000","last_became_peered":"2024-09-14T06:59:37.044304+0000","last_unstale":"2024-09-14T07:02:00.271596+0000","last_undegraded":"2024-09-14T07:02:00.271596+0000","last_fullsized":"2024-09-14T07:02:00.271596+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'679","last_scrub_stamp":"2024-09-14T07:02:00.271430+0000","last_deep_scrub":"152'679","last_deep_scrub_stamp":"2024-09-14T07:02:00.271430+0000","last_clean_scrub_stamp":"2024-09-14T07:02:00.271430+0000","objects_scrubbed":17,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:55:54.528168+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0011507799999999999,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":5024,"num_read_kb":10270,"num_write":682,"num_write_kb":6392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"152'416","reported_seq":6880,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:00.624382+0000","last_change":"2024-09-14T07:02:00.624382+0000","last_active":"2024-09-14T07:02:00.624382+0000","last_peered":"2024-09-14T07:02:00.624382+0000","last_clean":"2024-09-14T07:02:00.624382+0000","last_became_active":"2024-09-14T06:59:28.801224+0000","last_became_peered":"2024-09-14T06:59:28.801224+0000","last_unstale":"2024-09-14T07:02:00.624382+0000","last_undegraded":"2024-09-14T07:02:00.624382+0000","last_fullsized":"2024-09-14T07:02:00.624382+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'416","last_scrub_stamp":"2024-09-14T07:02:00.624283+0000","last_deep_scrub":"152'416","last_deep_scrub_stamp":"2024-09-14T07:02:00.624283+0000","last_clean_scrub_stamp":"2024-09-14T07:02:00.624283+0000","objects_scrubbed":24,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:52.476950+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0010952469999999999,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":7606,"num_read_kb":16770,"num_write":322,"num_write_kb":2846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"152'428","reported_seq":5217,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:59.247621+0000","last_change":"2024-09-14T07:01:59.247621+0000","last_active":"2024-09-14T07:01:59.247621+0000","last_peered":"2024-09-14T07:01:59.247621+0000","last_clean":"2024-09-14T07:01:59.247621+0000","last_became_active":"2024-09-14T07:01:43.462372+0000","last_became_peered":"2024-09-14T07:01:43.462372+0000","last_unstale":"2024-09-14T07:01:59.247621+0000","last_undegraded":"2024-09-14T07:01:59.247621+0000","last_fullsized":"2024-09-14T07:01:59.247621+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'428","last_scrub_stamp":"2024-09-14T07:01:59.247510+0000","last_deep_scrub":"152'428","last_deep_scrub_stamp":"2024-09-14T07:01:59.247510+0000","last_clean_scrub_stamp":"2024-09-14T07:01:59.247510+0000","objects_scrubbed":15,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:24:34.239499+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0022563269999999998,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":4902,"num_read_kb":8441,"num_write":353,"num_write_kb":3835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6621302,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"152'366","reported_seq":2812,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:59.238393+0000","last_change":"2024-09-14T07:01:59.238393+0000","last_active":"2024-09-14T07:01:59.238393+0000","last_peered":"2024-09-14T07:01:59.238393+0000","last_clean":"2024-09-14T07:01:59.238393+0000","last_became_active":"2024-09-14T07:01:36.304931+0000","last_became_peered":"2024-09-14T07:01:36.304931+0000","last_unstale":"2024-09-14T07:01:59.238393+0000","last_undegraded":"2024-09-14T07:01:59.238393+0000","last_fullsized":"2024-09-14T07:01:59.238393+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'366","last_scrub_stamp":"2024-09-14T07:01:59.238290+0000","last_deep_scrub":"152'366","last_deep_scrub_stamp":"2024-09-14T07:01:59.238290+0000","last_clean_scrub_stamp":"2024-09-14T07:01:59.238290+0000","objects_scrubbed":20,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:35:00.470497+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.0020075470000000002,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1776,"num_read_kb":3899,"num_write":305,"num_write_kb":4927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":143,"num_bytes_recovered":154159,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"152'809","reported_seq":5702,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:03.004696+0000","last_change":"2024-09-14T07:02:03.004696+0000","last_active":"2024-09-14T07:02:03.004696+0000","last_peered":"2024-09-14T07:02:03.004696+0000","last_clean":"2024-09-14T07:02:03.004696+0000","last_became_active":"2024-09-14T07:01:43.461464+0000","last_became_peered":"2024-09-14T07:01:43.461464+0000","last_unstale":"2024-09-14T07:02:03.004696+0000","last_undegraded":"2024-09-14T07:02:03.004696+0000","last_fullsized":"2024-09-14T07:02:03.004696+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'809","last_scrub_stamp":"2024-09-14T07:02:03.004561+0000","last_deep_scrub":"152'809","last_deep_scrub_stamp":"2024-09-14T07:02:03.004561+0000","last_clean_scrub_stamp":"2024-09-14T07:02:03.004561+0000","objects_scrubbed":20,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:06.882169+0000","scrub_duration":16,"objects_trimmed":1,"snaptrim_duration":0.0087910980000000007,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":6730,"num_read_kb":12616,"num_write":1057,"num_write_kb":7169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":14310721,"num_keys_recovered":36,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":289,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:00.036182+0000","last_change":"2024-09-14T07:02:00.036182+0000","last_active":"2024-09-14T07:02:00.036182+0000","last_peered":"2024-09-14T07:02:00.036182+0000","last_clean":"2024-09-14T07:02:00.036182+0000","last_became_active":"2024-09-14T06:59:20.104150+0000","last_became_peered":"2024-09-14T06:59:20.104150+0000","last_unstale":"2024-09-14T07:02:00.036182+0000","last_undegraded":"2024-09-14T07:02:00.036182+0000","last_fullsized":"2024-09-14T07:02:00.036182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:02:00.036093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:02:00.036093+0000","last_clean_scrub_stamp":"2024-09-14T07:02:00.036093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:40:38.330314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013176240000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"152'819","reported_seq":7050,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:00.520011+0000","last_change":"2024-09-14T07:02:00.520011+0000","last_active":"2024-09-14T07:02:00.520011+0000","last_peered":"2024-09-14T07:02:00.520011+0000","last_clean":"2024-09-14T07:02:00.520011+0000","last_became_active":"2024-09-14T06:59:28.808617+0000","last_became_peered":"2024-09-14T06:59:28.808617+0000","last_unstale":"2024-09-14T07:02:00.520011+0000","last_undegraded":"2024-09-14T07:02:00.520011+0000","last_fullsized":"2024-09-14T07:02:00.520011+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'819","last_scrub_stamp":"2024-09-14T07:02:00.519880+0000","last_deep_scrub":"152'819","last_deep_scrub_stamp":"2024-09-14T07:02:00.519880+0000","last_clean_scrub_stamp":"2024-09-14T07:02:00.519880+0000","objects_scrubbed":19,"log_size":819,"log_dups_size":0,"ondisk_log_size":819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:39:41.356810+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.0010359880000000001,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8046,"num_read_kb":9156,"num_write":887,"num_write_kb":5801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":289,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:58.584301+0000","last_change":"2024-09-14T07:01:58.584301+0000","last_active":"2024-09-14T07:01:58.584301+0000","last_peered":"2024-09-14T07:01:58.584301+0000","last_clean":"2024-09-14T07:01:58.584301+0000","last_became_active":"2024-09-14T06:59:20.104051+0000","last_became_peered":"2024-09-14T06:59:20.104051+0000","last_unstale":"2024-09-14T07:01:58.584301+0000","last_undegraded":"2024-09-14T07:01:58.584301+0000","last_fullsized":"2024-09-14T07:01:58.584301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:01:58.584188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:01:58.584188+0000","last_clean_scrub_stamp":"2024-09-14T07:01:58.584188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:39.755712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052221199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"152'526","reported_seq":11772,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:59.562537+0000","last_change":"2024-09-14T07:01:59.562537+0000","last_active":"2024-09-14T07:01:59.562537+0000","last_peered":"2024-09-14T07:01:59.562537+0000","last_clean":"2024-09-14T07:01:59.562537+0000","last_became_active":"2024-09-14T07:01:36.305222+0000","last_became_peered":"2024-09-14T07:01:36.305222+0000","last_unstale":"2024-09-14T07:01:59.562537+0000","last_undegraded":"2024-09-14T07:01:59.562537+0000","last_fullsized":"2024-09-14T07:01:59.562537+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'526","last_scrub_stamp":"2024-09-14T07:01:59.562423+0000","last_deep_scrub":"152'526","last_deep_scrub_stamp":"2024-09-14T07:01:59.562423+0000","last_clean_scrub_stamp":"2024-09-14T07:01:59.562423+0000","objects_scrubbed":16,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:53:53.274547+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001033576,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":12680,"num_read_kb":20099,"num_write":483,"num_write_kb":7950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":1892490,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":289,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:59.064405+0000","last_change":"2024-09-14T07:01:59.064405+0000","last_active":"2024-09-14T07:01:59.064405+0000","last_peered":"2024-09-14T07:01:59.064405+0000","last_clean":"2024-09-14T07:01:59.064405+0000","last_became_active":"2024-09-14T06:59:19.761071+0000","last_became_peered":"2024-09-14T06:59:19.761071+0000","last_unstale":"2024-09-14T07:01:59.064405+0000","last_undegraded":"2024-09-14T07:01:59.064405+0000","last_fullsized":"2024-09-14T07:01:59.064405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:01:59.064321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:01:59.064321+0000","last_clean_scrub_stamp":"2024-09-14T07:01:59.064321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:29:28.110267+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00083893699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"152'527","reported_seq":3974,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:59.587884+0000","last_change":"2024-09-14T07:01:59.587884+0000","last_active":"2024-09-14T07:01:59.587884+0000","last_peered":"2024-09-14T07:01:59.587884+0000","last_clean":"2024-09-14T07:01:59.587884+0000","last_became_active":"2024-09-14T07:01:47.490786+0000","last_became_peered":"2024-09-14T07:01:47.490786+0000","last_unstale":"2024-09-14T07:01:59.587884+0000","last_undegraded":"2024-09-14T07:01:59.587884+0000","last_fullsized":"2024-09-14T07:01:59.587884+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'527","last_scrub_stamp":"2024-09-14T07:01:59.587761+0000","last_deep_scrub":"152'527","last_deep_scrub_stamp":"2024-09-14T07:01:59.587761+0000","last_clean_scrub_stamp":"2024-09-14T07:01:59.587761+0000","objects_scrubbed":19,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:19:13.247305+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00070556600000000003,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":4047,"num_read_kb":4338,"num_write":428,"num_write_kb":4225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":813573,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":313,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:58.592774+0000","last_change":"2024-09-14T07:01:58.592774+0000","last_active":"2024-09-14T07:01:58.592774+0000","last_peered":"2024-09-14T07:01:58.592774+0000","last_clean":"2024-09-14T07:01:58.592774+0000","last_became_active":"2024-09-14T07:01:33.416397+0000","last_became_peered":"2024-09-14T07:01:33.416397+0000","last_unstale":"2024-09-14T07:01:58.592774+0000","last_undegraded":"2024-09-14T07:01:58.592774+0000","last_fullsized":"2024-09-14T07:01:58.592774+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:01:58.592691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:01:58.592691+0000","last_clean_scrub_stamp":"2024-09-14T07:01:58.592691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:25:51.130211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066409699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"152'592","reported_seq":10491,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:57.233424+0000","last_change":"2024-09-14T07:01:57.233424+0000","last_active":"2024-09-14T07:01:57.233424+0000","last_peered":"2024-09-14T07:01:57.233424+0000","last_clean":"2024-09-14T07:01:57.233424+0000","last_became_active":"2024-09-14T07:01:36.302844+0000","last_became_peered":"2024-09-14T07:01:36.302844+0000","last_unstale":"2024-09-14T07:01:57.233424+0000","last_undegraded":"2024-09-14T07:01:57.233424+0000","last_fullsized":"2024-09-14T07:01:57.233424+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'592","last_scrub_stamp":"2024-09-14T07:01:57.233313+0000","last_deep_scrub":"152'592","last_deep_scrub_stamp":"2024-09-14T07:01:57.233313+0000","last_clean_scrub_stamp":"2024-09-14T07:01:57.233313+0000","objects_scrubbed":19,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:25:35.167493+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00093736199999999996,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8151,"num_read_kb":12914,"num_write":603,"num_write_kb":8322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":286,"num_bytes_recovered":9390040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"152'832","reported_seq":8568,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:58.215664+0000","last_change":"2024-09-14T07:01:58.215664+0000","last_active":"2024-09-14T07:01:58.215664+0000","last_peered":"2024-09-14T07:01:58.215664+0000","last_clean":"2024-09-14T07:01:58.215664+0000","last_became_active":"2024-09-14T07:01:35.301421+0000","last_became_peered":"2024-09-14T07:01:35.301421+0000","last_unstale":"2024-09-14T07:01:58.215664+0000","last_undegraded":"2024-09-14T07:01:58.215664+0000","last_fullsized":"2024-09-14T07:01:58.215664+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'832","last_scrub_stamp":"2024-09-14T07:01:58.215542+0000","last_deep_scrub":"152'832","last_deep_scrub_stamp":"2024-09-14T07:01:58.215542+0000","last_clean_scrub_stamp":"2024-09-14T07:01:58.215542+0000","objects_scrubbed":19,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:13:08.965171+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.0027805429999999999,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":8837,"num_read_kb":11133,"num_write":1016,"num_write_kb":6151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":1769904,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"152'3","reported_seq":40,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:03.971873+0000","last_change":"2024-09-14T07:02:03.971873+0000","last_active":"2024-09-14T07:02:03.971873+0000","last_peered":"2024-09-14T07:02:03.971873+0000","last_clean":"2024-09-14T07:02:03.971873+0000","last_became_active":"2024-09-14T07:01:40.444161+0000","last_became_peered":"2024-09-14T07:01:40.444161+0000","last_unstale":"2024-09-14T07:02:03.971873+0000","last_undegraded":"2024-09-14T07:02:03.971873+0000","last_fullsized":"2024-09-14T07:02:03.971873+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'3","last_scrub_stamp":"2024-09-14T07:02:03.971739+0000","last_deep_scrub":"152'3","last_deep_scrub_stamp":"2024-09-14T07:02:03.971739+0000","last_clean_scrub_stamp":"2024-09-14T07:02:03.971739+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:24:09.124454+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0014065830000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.0","version":"152'489","reported_seq":22220,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:02.594692+0000","last_change":"2024-09-14T07:02:02.594692+0000","last_active":"2024-09-14T07:02:02.594692+0000","last_peered":"2024-09-14T07:02:02.594692+0000","last_clean":"2024-09-14T07:02:02.594692+0000","last_became_active":"2024-09-14T06:59:28.808309+0000","last_became_peered":"2024-09-14T06:59:28.808309+0000","last_unstale":"2024-09-14T07:02:02.594692+0000","last_undegraded":"2024-09-14T07:02:02.594692+0000","last_fullsized":"2024-09-14T07:02:02.594692+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'489","last_scrub_stamp":"2024-09-14T07:02:02.594587+0000","last_deep_scrub":"152'489","last_deep_scrub_stamp":"2024-09-14T07:02:02.594587+0000","last_clean_scrub_stamp":"2024-09-14T07:02:02.594587+0000","objects_scrubbed":12,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:05:25.321987+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.00089711299999999995,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":22342,"num_read_kb":44381,"num_write":481,"num_write_kb":17365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":288,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:59.909764+0000","last_change":"2024-09-14T07:01:59.909764+0000","last_active":"2024-09-14T07:01:59.909764+0000","last_peered":"2024-09-14T07:01:59.909764+0000","last_clean":"2024-09-14T07:01:59.909764+0000","last_became_active":"2024-09-14T06:59:19.763200+0000","last_became_peered":"2024-09-14T06:59:19.763200+0000","last_unstale":"2024-09-14T07:01:59.909764+0000","last_undegraded":"2024-09-14T07:01:59.909764+0000","last_fullsized":"2024-09-14T07:01:59.909764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:01:59.909680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:01:59.909680+0000","last_clean_scrub_stamp":"2024-09-14T07:01:59.909680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:32:45.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054069100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"152'408","reported_seq":2605,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:58.588711+0000","last_change":"2024-09-14T07:01:58.588711+0000","last_active":"2024-09-14T07:01:58.588711+0000","last_peered":"2024-09-14T07:01:58.588711+0000","last_clean":"2024-09-14T07:01:58.588711+0000","last_became_active":"2024-09-14T07:01:47.485661+0000","last_became_peered":"2024-09-14T07:01:47.485661+0000","last_unstale":"2024-09-14T07:01:58.588711+0000","last_undegraded":"2024-09-14T07:01:58.588711+0000","last_fullsized":"2024-09-14T07:01:58.588711+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'408","last_scrub_stamp":"2024-09-14T07:01:58.588601+0000","last_deep_scrub":"152'408","last_deep_scrub_stamp":"2024-09-14T07:01:58.588601+0000","last_clean_scrub_stamp":"2024-09-14T07:01:58.588601+0000","objects_scrubbed":19,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:58:01.245846+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.001602144,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":3106,"num_read_kb":12645,"num_write":311,"num_write_kb":7126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":1565893,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":289,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:57.083994+0000","last_change":"2024-09-14T07:01:57.083994+0000","last_active":"2024-09-14T07:01:57.083994+0000","last_peered":"2024-09-14T07:01:57.083994+0000","last_clean":"2024-09-14T07:01:57.083994+0000","last_became_active":"2024-09-14T06:59:19.754980+0000","last_became_peered":"2024-09-14T06:59:19.754980+0000","last_unstale":"2024-09-14T07:01:57.083994+0000","last_undegraded":"2024-09-14T07:01:57.083994+0000","last_fullsized":"2024-09-14T07:01:57.083994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T07:01:57.083893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T07:01:57.083893+0000","last_clean_scrub_stamp":"2024-09-14T07:01:57.083893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:03:55.085531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00080840800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"152'731","reported_seq":10710,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:57.627347+0000","last_change":"2024-09-14T07:01:57.627347+0000","last_active":"2024-09-14T07:01:57.627347+0000","last_peered":"2024-09-14T07:01:57.627347+0000","last_clean":"2024-09-14T07:01:57.627347+0000","last_became_active":"2024-09-14T07:00:30.291008+0000","last_became_peered":"2024-09-14T07:00:30.291008+0000","last_unstale":"2024-09-14T07:01:57.627347+0000","last_undegraded":"2024-09-14T07:01:57.627347+0000","last_fullsized":"2024-09-14T07:01:57.627347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'731","last_scrub_stamp":"2024-09-14T07:01:57.627245+0000","last_deep_scrub":"152'731","last_deep_scrub_stamp":"2024-09-14T07:01:57.627245+0000","last_clean_scrub_stamp":"2024-09-14T07:01:57.627245+0000","objects_scrubbed":21,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:02:54.978343+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.00074696400000000003,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":11440,"num_read_kb":36601,"num_write":924,"num_write_kb":12561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":5099646,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"152'2","reported_seq":311,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:58.064720+0000","last_change":"2024-09-14T07:01:58.064720+0000","last_active":"2024-09-14T07:01:58.064720+0000","last_peered":"2024-09-14T07:01:58.064720+0000","last_clean":"2024-09-14T07:01:58.064720+0000","last_became_active":"2024-09-14T07:00:27.521150+0000","last_became_peered":"2024-09-14T07:00:27.521150+0000","last_unstale":"2024-09-14T07:01:58.064720+0000","last_undegraded":"2024-09-14T07:01:58.064720+0000","last_fullsized":"2024-09-14T07:01:58.064720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'2","last_scrub_stamp":"2024-09-14T07:01:58.064654+0000","last_deep_scrub":"152'2","last_deep_scrub_stamp":"2024-09-14T07:01:58.064654+0000","last_clean_scrub_stamp":"2024-09-14T07:01:58.064654+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:51:24.808954+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0014726279999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"152'74","reported_seq":381,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:56.703019+0000","last_change":"2024-09-14T07:01:56.703019+0000","last_active":"2024-09-14T07:01:56.703019+0000","last_peered":"2024-09-14T07:01:56.703019+0000","last_clean":"2024-09-14T07:01:56.703019+0000","last_became_active":"2024-09-14T06:59:14.301017+0000","last_became_peered":"2024-09-14T06:59:14.301017+0000","last_unstale":"2024-09-14T07:01:56.703019+0000","last_undegraded":"2024-09-14T07:01:56.703019+0000","last_fullsized":"2024-09-14T07:01:56.703019+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'74","last_scrub_stamp":"2024-09-14T07:01:56.702943+0000","last_deep_scrub":"152'74","last_deep_scrub_stamp":"2024-09-14T07:01:56.702943+0000","last_clean_scrub_stamp":"2024-09-14T07:01:56.702943+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:43:12.611623+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"152'611","reported_seq":10113,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:58.923628+0000","last_change":"2024-09-14T07:01:58.923628+0000","last_active":"2024-09-14T07:01:58.923628+0000","last_peered":"2024-09-14T07:01:58.923628+0000","last_clean":"2024-09-14T07:01:58.923628+0000","last_became_active":"2024-09-14T07:00:53.610102+0000","last_became_peered":"2024-09-14T07:00:53.610102+0000","last_unstale":"2024-09-14T07:01:58.923628+0000","last_undegraded":"2024-09-14T07:01:58.923628+0000","last_fullsized":"2024-09-14T07:01:58.923628+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'611","last_scrub_stamp":"2024-09-14T07:01:58.923520+0000","last_deep_scrub":"152'611","last_deep_scrub_stamp":"2024-09-14T07:01:58.923520+0000","last_clean_scrub_stamp":"2024-09-14T07:01:58.923520+0000","objects_scrubbed":17,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:37:09.756901+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00097674599999999991,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":9511,"num_read_kb":20954,"num_write":534,"num_write_kb":10486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":4712244,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"152'786","reported_seq":6076,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:01.529462+0000","last_change":"2024-09-14T07:02:01.529462+0000","last_active":"2024-09-14T07:02:01.529462+0000","last_peered":"2024-09-14T07:02:01.529462+0000","last_clean":"2024-09-14T07:02:01.529462+0000","last_became_active":"2024-09-14T07:01:37.303369+0000","last_became_peered":"2024-09-14T07:01:37.303369+0000","last_unstale":"2024-09-14T07:02:01.529462+0000","last_undegraded":"2024-09-14T07:02:01.529462+0000","last_fullsized":"2024-09-14T07:02:01.529462+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'786","last_scrub_stamp":"2024-09-14T07:02:01.529335+0000","last_deep_scrub":"152'786","last_deep_scrub_stamp":"2024-09-14T07:02:01.529335+0000","last_clean_scrub_stamp":"2024-09-14T07:02:01.529335+0000","objects_scrubbed":17,"log_size":786,"log_dups_size":0,"ondisk_log_size":786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:28:54.823318+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0024576519999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":8449,"num_read_kb":18168,"num_write":805,"num_write_kb":7676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"152'1140","reported_seq":8439,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:05.950697+0000","last_change":"2024-09-14T07:02:05.950697+0000","last_active":"2024-09-14T07:02:05.950697+0000","last_peered":"2024-09-14T07:02:05.950697+0000","last_clean":"2024-09-14T07:02:05.950697+0000","last_became_active":"2024-09-14T06:59:28.798200+0000","last_became_peered":"2024-09-14T06:59:28.798200+0000","last_unstale":"2024-09-14T07:02:05.950697+0000","last_undegraded":"2024-09-14T07:02:05.950697+0000","last_fullsized":"2024-09-14T07:02:05.950697+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'1140","last_scrub_stamp":"2024-09-14T07:02:05.950574+0000","last_deep_scrub":"152'1140","last_deep_scrub_stamp":"2024-09-14T07:02:05.950574+0000","last_clean_scrub_stamp":"2024-09-14T07:02:05.950574+0000","objects_scrubbed":17,"log_size":1140,"log_dups_size":0,"ondisk_log_size":1140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:40:07.090649+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.00157712,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":10507,"num_read_kb":9740,"num_write":1539,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"152'366","reported_seq":3189,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:59.606991+0000","last_change":"2024-09-14T07:01:59.606991+0000","last_active":"2024-09-14T07:01:59.606991+0000","last_peered":"2024-09-14T07:01:59.606991+0000","last_clean":"2024-09-14T07:01:59.606991+0000","last_became_active":"2024-09-14T07:01:36.305298+0000","last_became_peered":"2024-09-14T07:01:36.305298+0000","last_unstale":"2024-09-14T07:01:59.606991+0000","last_undegraded":"2024-09-14T07:01:59.606991+0000","last_fullsized":"2024-09-14T07:01:59.606991+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'366","last_scrub_stamp":"2024-09-14T07:01:59.606898+0000","last_deep_scrub":"152'366","last_deep_scrub_stamp":"2024-09-14T07:01:59.606898+0000","last_clean_scrub_stamp":"2024-09-14T07:01:59.606898+0000","objects_scrubbed":14,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:01:51.880911+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00074805599999999998,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1936,"num_read_kb":8361,"num_write":285,"num_write_kb":3165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":1065606,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"152'473","reported_seq":5833,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:56.207960+0000","last_change":"2024-09-14T07:01:56.207960+0000","last_active":"2024-09-14T07:01:56.207960+0000","last_peered":"2024-09-14T07:01:56.207960+0000","last_clean":"2024-09-14T07:01:56.207960+0000","last_became_active":"2024-09-14T07:01:35.302427+0000","last_became_peered":"2024-09-14T07:01:35.302427+0000","last_unstale":"2024-09-14T07:01:56.207960+0000","last_undegraded":"2024-09-14T07:01:56.207960+0000","last_fullsized":"2024-09-14T07:01:56.207960+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'473","last_scrub_stamp":"2024-09-14T07:01:56.207878+0000","last_deep_scrub":"152'473","last_deep_scrub_stamp":"2024-09-14T07:01:56.207878+0000","last_clean_scrub_stamp":"2024-09-14T07:01:56.207878+0000","objects_scrubbed":13,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:24:15.071245+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.000724418,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":5156,"num_read_kb":11268,"num_write":417,"num_write_kb":10218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"152'461","reported_seq":2299,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:02.044821+0000","last_change":"2024-09-14T07:02:02.044821+0000","last_active":"2024-09-14T07:02:02.044821+0000","last_peered":"2024-09-14T07:02:02.044821+0000","last_clean":"2024-09-14T07:02:02.044821+0000","last_became_active":"2024-09-14T06:59:28.798219+0000","last_became_peered":"2024-09-14T06:59:28.798219+0000","last_unstale":"2024-09-14T07:02:02.044821+0000","last_undegraded":"2024-09-14T07:02:02.044821+0000","last_fullsized":"2024-09-14T07:02:02.044821+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'461","last_scrub_stamp":"2024-09-14T07:02:02.044700+0000","last_deep_scrub":"152'461","last_deep_scrub_stamp":"2024-09-14T07:02:02.044700+0000","last_clean_scrub_stamp":"2024-09-14T07:02:02.044700+0000","objects_scrubbed":20,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:28:02.178432+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.0015159500000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":1920,"num_read_kb":6604,"num_write":451,"num_write_kb":6928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"152'470","reported_seq":12427,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:58.245876+0000","last_change":"2024-09-14T07:01:58.245876+0000","last_active":"2024-09-14T07:01:58.245876+0000","last_peered":"2024-09-14T07:01:58.245876+0000","last_clean":"2024-09-14T07:01:58.245876+0000","last_became_active":"2024-09-14T07:00:31.293026+0000","last_became_peered":"2024-09-14T07:00:31.293026+0000","last_unstale":"2024-09-14T07:01:58.245876+0000","last_undegraded":"2024-09-14T07:01:58.245876+0000","last_fullsized":"2024-09-14T07:01:58.245876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'470","last_scrub_stamp":"2024-09-14T07:01:58.245712+0000","last_deep_scrub":"152'470","last_deep_scrub_stamp":"2024-09-14T07:01:58.245712+0000","last_clean_scrub_stamp":"2024-09-14T07:01:58.245712+0000","objects_scrubbed":20,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:19:21.829892+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.001728802,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":12086,"num_read_kb":54262,"num_write":423,"num_write_kb":18409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":8714381,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"152'535","reported_seq":4896,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:57.187807+0000","last_change":"2024-09-14T07:01:57.187807+0000","last_active":"2024-09-14T07:01:57.187807+0000","last_peered":"2024-09-14T07:01:57.187807+0000","last_clean":"2024-09-14T07:01:57.187807+0000","last_became_active":"2024-09-14T06:59:28.801931+0000","last_became_peered":"2024-09-14T06:59:28.801931+0000","last_unstale":"2024-09-14T07:01:57.187807+0000","last_undegraded":"2024-09-14T07:01:57.187807+0000","last_fullsized":"2024-09-14T07:01:57.187807+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'535","last_scrub_stamp":"2024-09-14T07:01:57.187738+0000","last_deep_scrub":"152'535","last_deep_scrub_stamp":"2024-09-14T07:01:57.187738+0000","last_clean_scrub_stamp":"2024-09-14T07:01:57.187738+0000","objects_scrubbed":28,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:34:40.774916+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.001321022,"stat_sum":{"num_bytes":0,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":28,"num_read":5747,"num_read_kb":11430,"num_write":424,"num_write_kb":4665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"152'456","reported_seq":4121,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:01:57.743270+0000","last_change":"2024-09-14T07:01:57.743270+0000","last_active":"2024-09-14T07:01:57.743270+0000","last_peered":"2024-09-14T07:01:57.743270+0000","last_clean":"2024-09-14T07:01:57.743270+0000","last_became_active":"2024-09-14T06:59:28.806090+0000","last_became_peered":"2024-09-14T06:59:28.806090+0000","last_unstale":"2024-09-14T07:01:57.743270+0000","last_undegraded":"2024-09-14T07:01:57.743270+0000","last_fullsized":"2024-09-14T07:01:57.743270+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'456","last_scrub_stamp":"2024-09-14T07:01:57.743180+0000","last_deep_scrub":"152'456","last_deep_scrub_stamp":"2024-09-14T07:01:57.743180+0000","last_clean_scrub_stamp":"2024-09-14T07:01:57.743180+0000","objects_scrubbed":16,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:38:01.377398+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00095644100000000004,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":3634,"num_read_kb":12358,"num_write":404,"num_write_kb":6155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"152'637","reported_seq":11126,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:00.537530+0000","last_change":"2024-09-14T07:02:00.537530+0000","last_active":"2024-09-14T07:02:00.537530+0000","last_peered":"2024-09-14T07:02:00.537530+0000","last_clean":"2024-09-14T07:02:00.537530+0000","last_became_active":"2024-09-14T07:01:47.490646+0000","last_became_peered":"2024-09-14T07:01:47.490646+0000","last_unstale":"2024-09-14T07:02:00.537530+0000","last_undegraded":"2024-09-14T07:02:00.537530+0000","last_fullsized":"2024-09-14T07:02:00.537530+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'637","last_scrub_stamp":"2024-09-14T07:02:00.537402+0000","last_deep_scrub":"152'637","last_deep_scrub_stamp":"2024-09-14T07:02:00.537402+0000","last_clean_scrub_stamp":"2024-09-14T07:02:00.537402+0000","objects_scrubbed":15,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:00:57.343423+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001614826,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":12269,"num_read_kb":24191,"num_write":663,"num_write_kb":8982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":13599677,"num_keys_recovered":35,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"152'744","reported_seq":7178,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:01.583344+0000","last_change":"2024-09-14T07:02:01.583344+0000","last_active":"2024-09-14T07:02:01.583344+0000","last_peered":"2024-09-14T07:02:01.583344+0000","last_clean":"2024-09-14T07:02:01.583344+0000","last_became_active":"2024-09-14T07:01:47.485532+0000","last_became_peered":"2024-09-14T07:01:47.485532+0000","last_unstale":"2024-09-14T07:02:01.583344+0000","last_undegraded":"2024-09-14T07:02:01.583344+0000","last_fullsized":"2024-09-14T07:02:01.583344+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'744","last_scrub_stamp":"2024-09-14T07:02:01.583223+0000","last_deep_scrub":"152'744","last_deep_scrub_stamp":"2024-09-14T07:02:01.583223+0000","last_clean_scrub_stamp":"2024-09-14T07:02:01.583223+0000","objects_scrubbed":19,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:39:23.059537+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.00071147799999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":8637,"num_read_kb":13164,"num_write":610,"num_write_kb":4987,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":1800074,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"152'627","reported_seq":6550,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-14T07:02:01.049947+0000","last_change":"2024-09-14T07:02:01.049947+0000","last_active":"2024-09-14T07:02:01.049947+0000","last_peered":"2024-09-14T07:02:01.049947+0000","last_clean":"2024-09-14T07:02:01.049947+0000","last_became_active":"2024-09-14T06:59:28.798979+0000","last_became_peered":"2024-09-14T06:59:28.798979+0000","last_unstale":"2024-09-14T07:02:01.049947+0000","last_undegraded":"2024-09-14T07:02:01.049947+0000","last_fullsized":"2024-09-14T07:02:01.049947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"152'627","last_scrub_stamp":"2024-09-14T07:02:01.049804+0000","last_deep_scrub":"152'627","last_deep_scrub_stamp":"2024-09-14T07:02:01.049804+0000","last_clean_scrub_stamp":"2024-09-14T07:02:01.049804+0000","objects_scrubbed":12,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:11:16.583284+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0019616249999999998,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":6772,"num_read_kb":10244,"num_write":478,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":579,"num_object_clones":0,"num_object_copies":1158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":575,"num_read":241540,"num_read_kb":530952,"num_write":22797,"num_write_kb":271683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3188,"num_bytes_recovered":84516668,"num_keys_recovered":235,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21005,"ondisk_log_size":21005,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":10262,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41284,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330556,"statfs":{"total":96636764160,"available":96594489344,"internally_reserved":0,"allocated":1822720,"data_stored":878219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63492,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308348,"statfs":{"total":96636764160,"available":96571748352,"internally_reserved":0,"allocated":1822720,"data_stored":878219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":63175111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59328,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312512,"statfs":{"total":96636764160,"available":96576012288,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64956,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306884,"statfs":{"total":96636764160,"available":96570249216,"internally_reserved":0,"allocated":1880064,"data_stored":1468587,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69056,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302784,"statfs":{"total":96636764160,"available":96566050816,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37180,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334660,"statfs":{"total":96636764160,"available":96598691840,"internally_reserved":0,"allocated":1880064,"data_stored":1468587,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542185,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55168,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316672,"statfs":{"total":96636764160,"available":96580272128,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45440,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94326400,"statfs":{"total":96636764160,"available":96590233600,"internally_reserved":0,"allocated":1818624,"data_stored":878200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":5131,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T07:02:19.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-14T07:02:19.632 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-14T07:02:19.633 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-14T07:02:19.633 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-14T07:02:19.633 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:19.674 INFO:tasks.ceph.osd.0:Stopped 2024-09-14T07:02:19.675 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-14T07:02:19.675 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:19.714 INFO:tasks.ceph.osd.1:Stopped 2024-09-14T07:02:19.714 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-14T07:02:19.714 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:19.791 INFO:tasks.ceph.osd.2:Stopped 2024-09-14T07:02:19.791 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-14T07:02:19.791 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:19.862 INFO:tasks.ceph.osd.3:Stopped 2024-09-14T07:02:19.862 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T07:02:19.862 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:19.902 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T07:02:19.903 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-14T07:02:19.903 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:19.984 INFO:tasks.ceph.osd.5:Stopped 2024-09-14T07:02:19.984 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-14T07:02:19.985 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:20.028 INFO:tasks.ceph.osd.6:Stopped 2024-09-14T07:02:20.028 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-14T07:02:20.028 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:20.073 INFO:tasks.ceph.osd.7:Stopped 2024-09-14T07:02:20.073 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-14T07:02:20.073 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-14T07:02:20.073 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:20.098 INFO:tasks.ceph.mgr.y:Stopped 2024-09-14T07:02:20.098 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-14T07:02:20.098 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:20.128 INFO:tasks.ceph.mgr.x:Stopped 2024-09-14T07:02:20.128 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-14T07:02:20.128 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-14T07:02:20.128 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:20.150 INFO:tasks.ceph.mon.a:Stopped 2024-09-14T07:02:20.151 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-14T07:02:20.151 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:20.202 INFO:tasks.ceph.mon.c:Stopped 2024-09-14T07:02:20.203 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-14T07:02:20.203 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T07:02:20.215 INFO:tasks.ceph.mon.b:Stopped 2024-09-14T07:02:20.215 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-14T07:02:20.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-14T07:02:20.298 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T07:02:20.298 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-14T07:02:20.512 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T07:02:20.512 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-14T07:02:20.609 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T07:02:20.610 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-14T07:02:20.667 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T07:02:20.667 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-14T07:02:20.717 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T07:02:20.717 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-14T07:02:20.928 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T07:02:20.929 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-14T07:02:21.008 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T07:02:21.009 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-14T07:02:21.066 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T07:02:21.066 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-14T07:02:21.123 INFO:tasks.ceph:Archiving mon data... 2024-09-14T07:02:21.124 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408/data/mon.a.tgz 2024-09-14T07:02:21.124 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-14T07:02:21.140 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.Ae7DgKk92Z 2024-09-14T07:02:21.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Ae7DgKk92Z 2024-09-14T07:02:21.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.Ae7DgKk92Z 2024-09-14T07:02:21.378 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.Ae7DgKk92Z is 385KB 2024-09-14T07:02:21.474 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.Ae7DgKk92Z 2024-09-14T07:02:21.490 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408/data/mon.c.tgz 2024-09-14T07:02:21.490 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-14T07:02:21.545 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.qLEhpA3wYr 2024-09-14T07:02:21.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.qLEhpA3wYr 2024-09-14T07:02:21.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.qLEhpA3wYr 2024-09-14T07:02:21.801 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.qLEhpA3wYr is 518KB 2024-09-14T07:02:21.872 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.qLEhpA3wYr 2024-09-14T07:02:21.887 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408/data/mon.b.tgz 2024-09-14T07:02:21.888 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-09-14T07:02:21.906 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.u3srShrRAf 2024-09-14T07:02:21.907 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.u3srShrRAf 2024-09-14T07:02:22.095 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.u3srShrRAf 2024-09-14T07:02:22.179 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.u3srShrRAf is 518KB 2024-09-14T07:02:22.251 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.u3srShrRAf 2024-09-14T07:02:22.266 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-14T07:02:22.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T07:02:22.269 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T07:02:22.349 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T07:02:22.384 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T07:02:22.385 INFO:tasks.ceph:Archiving crash dumps... 2024-09-14T07:02:22.386 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408/remote/smithi007/crash 2024-09-14T07:02:22.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T07:02:22.416 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408/remote/smithi189/crash 2024-09-14T07:02:22.417 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T07:02:22.448 INFO:tasks.ceph:Compressing logs... 2024-09-14T07:02:22.448 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T07:02:22.460 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T07:02:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-14T07:02:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51950.log 2024-09-14T07:02:22.527 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33097.log 2024-09-14T07:02:22.527 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-14T07:02:22.527 INFO:teuthology.orchestra.run.smithi189.stderr:gzip/var/log/ceph/ceph-client.admin.33097.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32977.log 2024-09-14T07:02:22.527 INFO:teuthology.orchestra.run.smithi189.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33097.log.gz 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.32977.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32977.log.gz 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33137.log 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33057.log 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.33137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33137.log.gz 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33177.log 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.33057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33057.log.gz 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.33177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33177.log.gz 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T07:02:22.528 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.6.log: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T07:02:22.529 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-14T07:02:22.529 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32931.log 2024-09-14T07:02:22.529 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-14T07:02:22.529 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.32931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32931.log.gz 2024-09-14T07:02:22.529 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33217.log 2024-09-14T07:02:22.529 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- 84.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T07:02:22.529 INFO:teuthology.orchestra.run.smithi189.stderr: /var/log/ceph/ceph-client.admin.33017.log 2024-09-14T07:02:22.529 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.33217.log: /var/log/ceph/ceph-mon.b.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33217.log.gz 2024-09-14T07:02:22.529 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.33017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33017.log.gz 2024-09-14T07:02:22.584 INFO:teuthology.orchestra.run.smithi189.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-14T07:02:22.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55081.log 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55123.log 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51950.log.gz 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54112.log 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55081.log.gz 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55123.log.gz 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52196.log 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54112.log.gz 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42616.log 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2024-09-14T07:02:22.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37784.log 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52196.log.gz 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37095.log 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42616.log.gz 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50940.log 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37784.log.gz 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49948.log 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37095.log.gz 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50940.log.gz 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40836.log 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49948.log.gz 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54703.log 2024-09-14T07:02:22.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51313.log 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40836.log.gz 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33508.log 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54703.log.gz 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53023.log 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51313.log.gz 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39178.log 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33508.log.gz 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53023.log.gz 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43477.log 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40420.log 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39178.log.gz 2024-09-14T07:02:22.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37146.log 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37230.log 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40420.log.gz 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37146.log.gz 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41836.log 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37230.log.gz 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40158.log 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49144.log 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41836.log.gz 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40158.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-14T07:02:22.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40158.log.gz 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49144.log.gz 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46746.log 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54236.log 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49084.log 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46746.log.gz 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54236.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54236.log.gz 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53856.log 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39136.log 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49084.log.gz 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53856.log.gz 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33413.log 2024-09-14T07:02:22.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50426.log 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39136.log.gz 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33413.log.gz 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54619.log 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40876.log 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50426.log.gz 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54619.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54619.log.gz -5 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54535.log 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52981.log 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40876.log.gz 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54535.log.gz 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44416.log 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39344.log 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52981.log.gz 2024-09-14T07:02:22.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44416.log.gz 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54745.log 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46352.log 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39344.log.gz 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51003.log 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54745.log.gz 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47286.log 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46352.log.gz 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51003.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.51003.log.gz 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37447.log 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53538.log 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47286.log.gz 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37447.log.gz 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53853.log 2024-09-14T07:02:22.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39218.log 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53538.log.gz 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53853.log.gz 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41316.log 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37783.log 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39218.log.gz 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53541.log 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41316.log.gz 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51065.log 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37783.log.gz 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37445.log 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53541.log.gz 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45473.log 2024-09-14T07:02:22.592 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51065.log.gz 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53065.log 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37445.log.gz 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37450.log 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45473.log.gz 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53065.log.gz 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39997.log 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37450.log.gz 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37454.log 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41635.log 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39997.log.gz 2024-09-14T07:02:22.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47611.log 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37454.log.gz 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55375.log 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41635.log.gz 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53536.log 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47611.log.gz 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49704.log 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55375.log.gz 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53536.log.gz 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44116.log 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43054.log 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49704.log.gz 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53275.log 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44116.log.gz 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51893.log 2024-09-14T07:02:22.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43054.log.gz 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53275.log.gz 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48052.log 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48570.log 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51893.log.gz 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48052.log.gz 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43655.log 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53855.log 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48570.log.gz 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43655.log.gz 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49184.log 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40559.log 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53855.log.gz 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49184.log.gz 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47366.log 2024-09-14T07:02:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38238.log 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40559.log.gz 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47366.log.gz 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41138.log 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38238.log.gz 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46908.log 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54577.log 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41138.log.gz 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52813.log 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46908.log.gz 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37451.log 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54577.log.gz 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51127.log 2024-09-14T07:02:22.596 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52813.log.gz 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43176.log 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37451.log.gz 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37443.log 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51127.log.gz 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44210.log 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43176.log.gz 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37443.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37443.log.gz 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37452.log 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44358.log 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44210.log.gz 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37452.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37452.log.gz -5 2024-09-14T07:02:22.597 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39996.log 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44358.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.44358.log.gz --verbose 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.48912.log 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37779.log 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39996.log.gz 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48912.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.48912.log.gz 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.50204.log 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52729.log 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37779.log.gz 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54493.log 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50204.log.gz 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46392.log 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52729.log.gz 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54493.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.54493.log.gz /var/log/ceph/ceph-client.admin.54174.log 2024-09-14T07:02:22.598 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53530.log 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46392.log.gz 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41754.log 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54174.log.gz 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54113.log 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53530.log.gz 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41754.log.gz 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55333.log 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50822.log 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54113.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54113.log.gz 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55333.log.gz 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48712.log 2024-09-14T07:02:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43534.log 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50822.log.gz 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54451.log 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48712.log.gz 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48911.log 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43534.log.gz 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54451.log.gz 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45956.log 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52415.log 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48911.log.gz 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45956.log.gz 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48112.log 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36925.log 2024-09-14T07:02:22.600 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52415.log.gz 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48112.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48112.log.gz 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53532.log 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39880.log 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36925.log.gz 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54409.log 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53532.log.gz 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48012.log 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39880.log.gz 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44697.log 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54409.log.gz 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48012.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48012.log.gz 2024-09-14T07:02:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54955.log 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44697.log.gz 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51189.log 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54955.log.gz 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55417.log 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54172.log 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51189.log.gz 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52771.log 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55417.log.gz 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41796.log 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54172.log.gz 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44754.log 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52771.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52771.log.gz 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51751.log 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41796.log.gz 2024-09-14T07:02:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44754.log.gz 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51554.log 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38349.log 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51751.log.gz 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53317.log 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51554.log.gz 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.38349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55249.log 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54997.log 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53317.log.gz 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.0.38349.log.gz 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55457.log 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55249.log.gz 2024-09-14T07:02:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47488.log 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54997.log.gz 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55457.log.gz 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37188.log 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52855.log 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47488.log.gz 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39642.log 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37188.log.gz 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43116.log 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52855.log.gz 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39642.log.gz 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45053.log 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54787.log 2024-09-14T07:02:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43116.log.gz 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45053.log.gz 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52514.log 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43873.log 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54787.log.gz 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52514.log.gz 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41712.log 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39019.log 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43873.log.gz 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41712.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41712.log.gz -- 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.42874.log 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53857.log 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39019.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39019.log.gz 2024-09-14T07:02:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42874.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz -5 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53881.log 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53857.log.gz 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53881.log.gz 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50483.log 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37785.log 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.50483.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.50483.log.gz -- 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.48254.log 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37785.log: /var/log/ceph/ceph-client.admin.46053.log 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37785.log.gz 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48254.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48254.log.gz 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49947.log 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45176.log 2024-09-14T07:02:22.606 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49947.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49947.log.gz 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38418.log 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42376.log 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45176.log.gz 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50603.log 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38418.log.gz 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42376.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42376.log.gz 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53149.log 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53776.log 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50603.log.gz 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54871.log 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53149.log.gz 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53776.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53776.log.gz 2024-09-14T07:02:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51512.log 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54871.log.gz 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41095.log 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: gzip/var/log/ceph/ceph-client.admin.51512.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51512.log.gz 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.37986.log 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41095.log.gz 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33117.log 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.37986.log: --verbose -- /var/log/ceph/ceph-client.admin.41898.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37986.log.gz 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33117.log.gz 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49622.log 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.41898.log: --verbose -- /var/log/ceph/ceph-client.admin.40081.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41898.log.gz 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49622.log.gz 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43115.log 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.40081.log: 0.0% --verbose -- /var/log/ceph/ceph.tmp-client.admin.27288.log 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40081.log.gz 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43115.log.gz 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55039.log 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.27288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37787.log 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27288.log.gz 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55039.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55039.log.gz 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.51992.log 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37787.log.gz 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53233.log 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.51992.log: -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51992.log.gz 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.53191.log 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53233.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53233.log.gz 2024-09-14T07:02:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38112.log 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.53191.log: --verbose -- /var/log/ceph/ceph-client.admin.55207.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53191.log.gz 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38112.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38112.log.gz 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54661.log 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.55207.log: --verbose -- 0.0% /var/log/ceph/ceph-client.admin.40995.log 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55207.log.gz 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55165.log 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54661.log.gz 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.40995.log: --verbose -- /var/log/ceph/ceph-client.admin.54829.log 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40995.log.gz 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55165.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45575.log 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55165.log.gz 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54829.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-14T07:02:22.610 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54829.log.gz 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.45575.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54173.log 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45575.log.gz 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49806.log 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mgr.y.log: gzip/var/log/ceph/ceph-client.admin.54173.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45635.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54173.log.gz 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49806.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39260.log 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40496.log 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39260.log: /var/log/ceph/ceph-client.admin.45635.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39260.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45635.log.gz 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45753.log 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51333.log 2024-09-14T07:02:22.611 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40496.log.gz 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45753.log.gz 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43959.log 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51333.log.gz 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43959.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43959.log.gz 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52687.log 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52939.log 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52687.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50862.log 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52687.log.gz 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38737.log 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50862.log: /var/log/ceph/ceph-client.admin.52939.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50862.log.gz 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52939.log.gz 2024-09-14T07:02:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38070.log 91.4% 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38737.log.gz 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52897.log 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47149.log/var/log/ceph/ceph-client.admin.38070.log: 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38070.log.gz 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52897.log.gz 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47673.log 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47149.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47149.log.gz 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53534.log 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47673.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52598.log 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47673.log.gz 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.53534.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38154.log 2024-09-14T07:02:22.613 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53534.log.gz 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52598.log: /var/log/ceph/ceph-client.admin.54073.log 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52598.log.gz 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38154.log.gz 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49010.log 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54073.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.55291.log 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54073.log.gz 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49265.log 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49010.log.gz 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.55291.log: --verbose -- /var/log/ceph/ceph-client.admin.42494.log 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55291.log.gz 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43012.log 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49265.log.gz 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.42494.log: -- /var/log/ceph/ceph-client.admin.38196.log 2024-09-14T07:02:22.614 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42494.log.gz 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43012.log.gz 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37650.log 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.38196.log: --verbose -- 0.0% /var/log/ceph/ceph-client.admin.39601.log 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38196.log.gz 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37650.log.gz 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51251.log 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39601.log.gz 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37789.log 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51251.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51251.log.gz 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53539.log 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37789.log.gz 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52136.log 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53539.log.gz 2024-09-14T07:02:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49564.log 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52136.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52136.log.gz 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.44916.log 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49564.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49564.log.gz 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42177.log 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44916.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44916.log.gz 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52645.log 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42177.log.gz 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48772.log 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52645.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52645.log.gz 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.38028.log 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48772.log.gz 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43872.log 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38028.log.gz 2024-09-14T07:02:22.616 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45135.log 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43872.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.52034.log 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43872.log.gz 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45135.log.gz 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49523.log 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53359.log/var/log/ceph/ceph-client.admin.52034.log: 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52034.log.gz 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49523.log.gz 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38794.log 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50087.log 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53359.log.gz 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38794.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38794.log.gz 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.52472.log 2024-09-14T07:02:22.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50087.log.gz 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53777.log 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.52472.log: --verbose -- /var/log/ceph/ceph-client.admin.37453.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52472.log.gz 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53777.log.gz 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52556.log 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.37453.log: --verbose -- /var/log/ceph/ceph-client.admin.53540.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37453.log.gz 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52556.log.gz 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41413.log 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.53540.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53540.log.gz 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.54913.log 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41413.log.gz 2024-09-14T07:02:22.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37790.log 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.54913.log: --verbose -- /var/log/ceph/ceph-client.admin.46194.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54913.log.gz 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37790.log.gz 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53107.log 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.46194.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45474.log 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46194.log.gz 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53107.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53107.log.gz 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.37070.log 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45474.log.gz 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53772.log 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37070.log.gz 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37272.log 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53772.log.gz 2024-09-14T07:02:22.619 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47713.log 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37272.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37272.log.gz 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.46691.log 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47713.log.gz 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46691.log.gz 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38300.log 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47067.log 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42713.log 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38300.log.gz 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47067.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48234.log 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47067.log.gz 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42713.log.gz 2024-09-14T07:02:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48832.log 2024-09-14T07:02:22.621 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38420.log 2024-09-14T07:02:22.621 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48234.log.gz 2024-09-14T07:02:22.621 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48832.log.gz 2024-09-14T07:02:22.621 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38420.log.gz 2024-09-14T07:02:22.673 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-14T07:02:22.692 INFO:teuthology.orchestra.run.smithi189.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-14T07:02:22.789 INFO:teuthology.orchestra.run.smithi007.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-14T07:02:22.812 INFO:teuthology.orchestra.run.smithi189.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-14T07:02:23.634 INFO:teuthology.orchestra.run.smithi189.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-14T07:02:24.952 INFO:teuthology.orchestra.run.smithi007.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-14T07:02:25.826 INFO:teuthology.orchestra.run.smithi007.stderr: 95.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-14T07:02:25.934 INFO:teuthology.orchestra.run.smithi007.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-14T07:02:26.429 INFO:teuthology.orchestra.run.smithi189.stderr: 96.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-14T07:02:26.930 INFO:teuthology.orchestra.run.smithi189.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-14T07:02:26.932 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-09-14T07:02:26.932 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m4.428s 2024-09-14T07:02:26.932 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m9.605s 2024-09-14T07:02:26.932 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m0.394s 2024-09-14T07:02:27.097 INFO:teuthology.orchestra.run.smithi007.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-14T07:02:27.099 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-14T07:02:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m4.626s 2024-09-14T07:02:27.099 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m13.701s 2024-09-14T07:02:27.100 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.643s 2024-09-14T07:02:27.100 INFO:tasks.ceph:Archiving logs... 2024-09-14T07:02:27.100 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408/remote/smithi007/log 2024-09-14T07:02:27.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T07:02:28.884 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408/remote/smithi189/log 2024-09-14T07:02:28.885 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T07:02:30.054 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-14T07:02:30.063 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-14T07:02:30.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T07:02:30.066 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T07:02:30.158 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-14T07:02:30.166 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-14T07:02:30.166 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T07:02:30.169 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T07:02:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-14T07:02:30.187 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-09-14T07:02:30.189 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T07:02:30.190 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-14T07:02:30.190 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 -683us[ -687us] +/- 50ms 2024-09-14T07:02:30.190 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 +815us[ +815us] +/- 61ms 2024-09-14T07:02:30.190 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +31us[ +31us] +/- 78ms 2024-09-14T07:02:30.190 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T07:02:30.191 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T07:02:30.191 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-09-14T07:02:30.191 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 -673us[ -672us] +/- 50ms 2024-09-14T07:02:30.192 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 +807us[ +807us] +/- 61ms 2024-09-14T07:02:30.192 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +42us[ +42us] +/- 78ms 2024-09-14T07:02:30.192 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T07:02:30.193 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-14T07:02:30.201 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-14T07:02:30.201 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-14T07:02:30.234 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-14T07:02:30.318 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-14T07:02:30.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T07:02:30.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:02:30.425 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T07:02:30.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:02:30.456 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-14T07:02:30.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-14T07:02:30.508 INFO:teuthology.task.internal:Duration was 625.184066 seconds 2024-09-14T07:02:30.509 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-14T07:02:30.543 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-14T07:02:30.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T07:02:30.545 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T07:02:30.586 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T07:02:30.587 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T07:02:30.825 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-14T07:02:30.825 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-09-14T07:02:30.825 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T07:02:30.848 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-09-14T07:02:30.848 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T07:02:30.888 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-14T07:02:30.888 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T07:02:30.890 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T07:02:30.954 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-14T07:02:30.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T07:02:30.959 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T07:02:31.105 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-14T07:02:31.115 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-14T07:02:31.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T07:02:31.141 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T07:02:31.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-14T07:02:31.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:02:31.184 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:02:31.205 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-09-14T07:02:31.228 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-09-14T07:02:31.253 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:02:31.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:02:31.285 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T07:02:31.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T07:02:31.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-14T07:02:31.318 INFO:teuthology.task.internal:Transferring archived files... 2024-09-14T07:02:31.319 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408/remote/smithi007 2024-09-14T07:02:31.319 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T07:02:31.366 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900408/remote/smithi189 2024-09-14T07:02:31.366 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T07:02:31.399 INFO:teuthology.task.internal:Removing archive directory... 2024-09-14T07:02:31.399 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T07:02:31.401 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T07:02:31.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-14T07:02:31.462 INFO:teuthology.task.internal:Not uploading archives. 2024-09-14T07:02:31.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-14T07:02:31.484 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-14T07:02:31.484 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T07:02:31.487 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T07:02:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 07:02 /home/ubuntu/cephtest 2024-09-14T07:02:31.505 INFO:teuthology.orchestra.run.smithi189.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 07:02 /home/ubuntu/cephtest 2024-09-14T07:02:31.506 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-14T07:02:31.517 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-14T07:02:31.590 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} duration: 625.1840662956238 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-14T07:02:31.590 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T07:02:31.703 INFO:teuthology.run:pass